Artificial Intelligence and Robotics Commons^™

Kodai: Framework Towards Data Augmentation Of Large Language Models In Machine Learning, Rick Rejeleene

Theses and Dissertations

Machine Learning is rapidly advancing at an incredible pace due to an increase in computational size, and availability of data. Data is the cornerstone of machine learning algorithms. Information quality has a profound impact on the performance of machine learning systems. Supervised, Unsupervised, and Reinforcement learning are three major ways of performing machine learning. Self-supervised learning, part of unsupervised learning, has made breakthroughs in engineering and research by using Large Language Models (LLM). LLM are a type of neural network, used for language understanding and generation. Recently, LLM has taken a distinct lead by demonstrating state-of-the-art capabilities in natural language …

Emergent Ai, Jillian A. Bick

CAFE Symposium 2024

For many years, artificial intelligence (AI) was considered to be limited in its abilities due to being confined to a pre-defined set of data. Currently, however, AI models have grown in complexity and size, leading to some previously impossible behaviors. These behaviors, known as "emergent AI behaviors," are unpredictable and not pre-programmed. Their existence suggests that AI is expanding in adaptability and may one day rival human intelligence. Media often portrays AI as having emotions and having the ability to operate autonomously, but what behaviors are AI really capable of?

Cybernetics: How It Compares To Science-Fiction And Future Possibilities, Anindo Majumder

CAFE Symposium 2024

Cybernetics is a branch of science that studies how information is communicated in machines and electronic equipment compared to how information is communicated in the brain and nervous system. It also relates to the theory of automatic control and physiology, particularly the physiology of the nervous system. Usage of cybernetics is very popular in various science-fiction medium. This naturally leads one to be curious if its depictions might turn into reality one day. This research paper delves into the growth of cybernetics since its inception, current applications of cybernetics, and what the future might hold.

A Comparative Study Of Responses To Retina Questions From Either Experts, Expert-Edited Large Language Models, Or Expert-Edited Large Language Models Alone, Prashant D. Tailor, Lauren A. Dalvin, John J. Chen, Raymond Iezzi, Timothy W. Olsen, Brittni A. Scruggs, Andrew J. Barkmeier, Sophie J. Bakri, Edwin H. Ryan, Peter H. Tang, D. Wilkin Parke, Peter Belin, Jayanth Sridhar, David Xu, Ajay E. Kuriyan, Yoshihiro Yonekawa, Matthew R. Starr

Wills Eye Hospital Papers

OBJECTIVE: To assess the quality, empathy, and safety of expert edited large language model (LLM), human expert created, and LLM responses to common retina patient questions.

DESIGN: Randomized, masked multicenter study.

PARTICIPANTS: Twenty-one common retina patient questions were randomly assigned among 13 retina specialists.

METHODS: Each expert created a response (Expert) and then edited a LLM (ChatGPT-4)-generated response to that question (Expert + artificial intelligence [AI]), timing themselves for both tasks. Five LLMs (ChatGPT-3.5, ChatGPT-4, Claude 2, Bing, and Bard) also generated responses to each question. The original question along with anonymized and randomized Expert + AI, Expert, and LLM …

Conic Challenge: Pushing The Frontiers Of Nuclear Detection, Segmentation, Classification And Counting, Simon Graham, Quoc Dang Vu, Mostafa Jahanifar, Martin Weigert, Uwe Schmidt, Wenhua Zhang, Jun Zhang, Sen Yang, Jinxi Xiang, Xiyue Wang, Josef Lorenz Rumberger, Elias Baumann, Peter Hirsch, Lihao Liu, Chenyang Hong, Angelica I. Aviles-Rivero, Ayushi Jain, Heeyoung Ahn, Yiyu Hong, Hussam Azzuni, Min Xu

Computer Vision Faculty Publications

Nuclear detection, segmentation and morphometric profiling are essential in helping us further understand the relationship between histology and patient outcome. To drive innovation in this area, we setup a community-wide challenge using the largest available dataset of its kind to assess nuclear segmentation and cellular composition. Our challenge, named CoNIC, stimulated the development of reproducible algorithms for cellular recognition with real-time result inspection on public leaderboards. We conducted an extensive post-challenge analysis based on the top-performing models using 1,658 whole-slide images of colon tissue. With around 700 million detected nuclei per model, associated features were used for dysplasia grading and …

Using Natural Language Processing And Patient Journey Clustering For Temporal Phenotyping Of Antimicrobial Therapies For Cat Bite Abscesses, Brian Hur, Karin M. Verspoor, Timothy Baldwin, Laura Y. Hardefeldt, Caitlin Pfeiffer, Caroline Mansfield, Riati Scarborough, James R. Gilkerson

Natural Language Processing Faculty Publications

Background: Temporal phenotyping of patient journeys, which capture the common sequence patterns of interventions in the treatment of a specific condition, is useful to support understanding of antimicrobial usage in veterinary patients. Identifying and describing these phenotypes can inform antimicrobial stewardship programs designed to fight antimicrobial resistance, a major health crisis affecting both humans and animals, in which veterinarians have an important role to play. Objective: This research proposes a framework for extracting temporal phenotypes of patient journeys from clinical practice data through the application of natural language processing (NLP) and unsupervised machine learning (ML) techniques, using cat bite abscesses …

Deep Learning-Based Human Action Understanding In Videos, Elahe Vahdani

Dissertations, Theses, and Capstone Projects

The understanding of human actions in videos holds immense potential for technological advancement and societal betterment. This thesis explores fundamental aspects of this field, including action recognition in trimmed clips and action localization in untrimmed videos. Trimmed videos contain only one action instance, with moments before or after the action excluded from the video. However, the majority of videos captured in unconstrained environments, often referred to as untrimmed videos, are naturally unsegmented. Untrimmed videos are typically lengthy and may encompass multiple action instances, along with the moments preceding or following each action, as well as transitions between actions. In the …

Glop: Learning Global Partition And Local Construction For Solving Large-Scale Routing Problems In Real-Time, Haoran Ye, Jiarui Wang, Helan Liang, Zhiguang Cao, Yong Li, Fanzhang Li

Research Collection School Of Computing and Information Systems

The recent end-to-end neural solvers have shown promise for small-scale routing problems but suffered from limited real-time scaling-up performance. This paper proposes GLOP (Global and Local Optimization Policies), a unified hierarchical framework that efficiently scales toward large-scale routing problems. GLOP partitions large routing problems into Travelling Salesman Problems (TSPs) and TSPs into Shortest Hamiltonian Path Problems. For the first time, we hybridize non-autoregressive neural heuristics for coarse-grained problem partitions and autoregressive neural heuristics for fine-grained route constructions, leveraging the scalability of the former and the meticulousness of the latter. Experimental results show that GLOP achieves competitive and state-of-the-art real-time performance …

Public Acceptance Of Using Artificial Intelligence-Assisted Weight Management Apps In High-Income Southeast Asian Adults With Overweight And Obesity: A Cross-Sectional Study, Han Shi Jocelyn Chew, Palakorn Achananuparp, Palakorn Achananuparp, Nicholas W. S. Chew, Yip Han Chin, Yujia Gao, Bok Yan Jimmy So, Asim Shabbir, Ee-Peng Lim, Kee Yuan Ngiam

Research Collection School Of Computing and Information Systems

Introduction: With in increase in interest to incorporate artificial intelligence (AI) into weight management programs, we aimed to examine user perceptions of AI-based mobile apps for weight management in adults with overweight and obesity. Methods: 280 participants were recruited between May and November 2022. Participants completed a questionnaire on sociodemographic profiles, Unified Theory of Acceptance and Use of Technology 2 (UTAUT2), and Self-Regulation of Eating Behavior Questionnaire. Structural equation modeling was performed using R. Model fit was tested using maximum-likelihood generalized unweighted least squares. Associations between influencing factors were analyzed using correlation and linear regression. Results: 271 participant responses were …

Delving Into Multimodal Prompting For Fine-Grained Visual Classification, Xin Jiang, Hao Tang, Junyao Gao, Xiaoyu Du, Shengfeng He, Zechao Li

Research Collection School Of Computing and Information Systems

Fine-grained visual classification (FGVC) involves categorizing fine subdivisions within a broader category, which poses challenges due to subtle inter-class discrepancies and large intra-class variations. However, prevailing approaches primarily focus on uni-modal visual concepts. Recent advancements in pre-trained vision-language models have demonstrated remarkable performance in various high-level vision tasks, yet the applicability of such models to FGVC tasks remains uncertain. In this paper, we aim to fully exploit the capabilities of cross-modal description to tackle FGVC tasks and propose a novel multimodal prompting solution, denoted as MP-FGVC, based on the contrastive language-image pertaining (CLIP) model. Our MP-FGVC comprises a multimodal prompts …

Leveraging Llms And Generative Models For Interactive Known-Item Video Search, Zhixin Ma, Jiaxin Wu, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

While embedding techniques such as CLIP have considerably boosted search performance, user strategies in interactive video search still largely operate on a trial-and-error basis. Users are often required to manually adjust their queries and carefully inspect the search results, which greatly rely on the users’ capability and proficiency. Recent advancements in large language models (LLMs) and generative models offer promising avenues for enhancing interactivity in video retrieval and reducing the personal bias in query interpretation, particularly in the known-item search. Specifically, LLMs can expand and diversify the semantics of the queries while avoiding grammar mistakes or the language barrier. In …

Mitigating Fine-Grained Hallucination By Fine-Tuning Large Vision-Language Models With Caption Rewrites, Lei Wang, Jiabang He, Shenshen Li, Ning Liu, Ee-Peng Lim

Research Collection School Of Computing and Information Systems

Large language models (LLMs) have shown remarkable performance in natural language processing (NLP) tasks. To comprehend and execute diverse human instructions over image data, instruction-tuned large vision-language models (LVLMs) have been introduced. However, LVLMs may suffer from different types of object hallucinations. Nevertheless, LVLMs are evaluated for coarse-grained object hallucinations only (i.e., generated objects non-existent in the input image). The fine-grained object attributes and behaviors non-existent in the image may still be generated but not measured by the current evaluation methods. In this paper, we thus focus on reducing fine-grained hallucinations of LVLMs. We propose ReCaption, a framework that consists …

Transition-Informed Reinforcement Learning For Large-Scale Stackelberg Mean-Field Games., Pengdeng Li, Runsheng Yu, Xinrun Wang, Bo An

Research Collection School Of Computing and Information Systems

Many real-world scenarios including fleet management and Ad auctions can be modeled as Stackelberg mean-field games (SMFGs) where a leader aims to incentivize a large number of homogeneous self-interested followers to maximize her utility. Existing works focus on cases with a small number of heterogeneous followers, e.g., 5-10, and suffer from scalability issue when the number of followers increases. There are three major challenges in solving large-scale SMFGs: i) classical methods based on solving differential equations fail as they require exact dynamics parameters, ii) learning by interacting with environment is data-inefficient, and iii) complex interaction between the leader and followers …

Earnhft: Efficient Hierarchical Reinforcement Learning For High Frequency Trading, Molei Qin, Shuo Sun, Wentao Zhang, Haochong Xia, Xinrun Wang, Bo An

Research Collection School Of Computing and Information Systems

High-frequency trading (HFT) is using computer algorithms to make trading decisions in short time scales (e.g., second-level), which is widely used in the Cryptocurrency (Crypto) market, (e.g., Bitcoin). Reinforcement learning (RL) in financial research has shown stellar performance on many quantitative trading tasks. However, most methods focus on low-frequency trading, e.g., day-level, which cannot be directly applied to HFT because of two challenges. First, RL for HFT involves dealing with extremely long trajectories (e.g., 2.4 million steps per month), which is hard to optimize and evaluate. Second, the dramatic price fluctuations and market trend changes of Crypto make existing algorithms …

Market-Gan: Adding Control To Financial Market Data Generation With Semantic Context, Haochong Xia, Shuo Sun, Xinrun Wang, Bo An

Research Collection School Of Computing and Information Systems

Financial simulators play an important role in enhancing forecasting accuracy, managing risks, and fostering strategic financial decision-making. Despite the development of financial market simulation methodologies, existing frameworks often struggle with adapting to specialized simulation context. We pinpoint the challenges as i) current financial datasets do not contain context labels; ii) current techniques are not designed to generate financial data with context as control, which demands greater precision compared to other modalities; iii) the inherent difficulties in generating context-aligned, high-fidelity data given the non-stationary, noisy nature of financial data. To address these challenges, our contributions are: i) we proposed the Contextual …

Machine Learning For Refining Knowledge Graphs: A Survey, Budhitama Subagdja, D. Shanthoshigaa, Zhaoxia Wang, Ah-Hwee Tan

Research Collection School Of Computing and Information Systems

Knowledge graph (KG) refinement refers to the process of filling in missing information, removing redundancies, and resolving inconsistencies in knowledge graphs. With the growing popularity of KG in various domains, many techniques involving machine learning have been applied, but there is no survey dedicated to machine learning-based KG refinement yet. Based on a novel framework following the KG refinement process, this paper presents a survey of machine learning approaches to KG refinement according to the kind of operations in KG refinement, the training datasets, mode of learning, and process multiplicity. Furthermore, the survey aims to provide broad practical insights into …

Handling Long And Richly Constrained Tasks Through Constrained Hierarchical Reinforcement Learning, Yuxiao Lu, Arunesh Sinha, Pradeep Varakantham

Research Collection School Of Computing and Information Systems

Safety in goal directed Reinforcement Learning (RL) settings has typically been handled through constraints over trajectories and have demonstrated good performance in primarily short horizon tasks. In this paper, we are specifically interested in the problem of solving temporally extended decision making problems such as robots cleaning different areas in a house while avoiding slippery and unsafe areas (e.g., stairs) and retaining enough charge to move to a charging dock; in the presence of complex safety constraints. Our key contribution is a (safety) Constrained Search with Hierarchical Reinforcement Learning (CoSHRL) mechanism that combines an upper level constrained search agent (which …

Predictive Algorithm For Surgery Recommendation In Thoracolumbar Burst Fractures Without Neurological Deficits, Charlotte Dandurand, Nader Fallah, Cumhur F. Öner, Richard J. Bransford, Klaus Schnake, Alex R. Vaccaro, Lorin M. Benneker, Emiliano Vialle, Gregory D. Schroeder, Shanmuganathan Rajasekaran, Mohammad El-Skarkawi, Rishi M. Kanna, Mohamed Aly, Martin Holas, Jose A. Canseco, Sander Muijs, Eugen Cezar Popescu, Jin Wee Tee, Gaston Camino-Willhuber, Andrei Fernandes Joaquim, Ory Keynan, Harvinder Singh Chhabra, Sebastian Bigdon, Ulrich Spiegel, Marcel F. Dvorak

Department of Orthopaedic Surgery Faculty Papers

STUDY DESIGN: Predictive algorithm via decision tree.

OBJECTIVES: Artificial intelligence (AI) remain an emerging field and have not previously been used to guide therapeutic decision making in thoracolumbar burst fractures. Building such models may reduce the variability in treatment recommendations. The goal of this study was to build a mathematical prediction rule based upon radiographic variables to guide treatment decisions.

METHODS: Twenty-two surgeons from the AO Knowledge Forum Trauma reviewed 183 cases from the Spine TL A3/A4 prospective study (classification, degree of certainty of posterior ligamentous complex (PLC) injury, use of M1 modifier, degree of comminution, treatment recommendation). Reviewers' regions …

Artificial Intelligence Model Predicts Sudden Cardiac Arrest Manifesting With Pulseless Electric Activity Versus Ventricular Fibrillation, Lauri Holmstrom, Bryan Bednarski, Harpriya Chugh, Habiba Aziz, Hoang Nhat Pham, Arayik Sargsyan, Audrey Uy-Evanado, Damini Dey, Angelo Salvucci, Jonathan Jui, Kyndaron Reinier, Piotr J Slomka, Sumeet S Chugh

Faculty, Staff and Student Publications

BACKGROUND: There is no specific treatment for sudden cardiac arrest (SCA) manifesting as pulseless electric activity (PEA) and survival rates are low; unlike ventricular fibrillation (VF), which is treatable by defibrillation. Development of novel treatments requires fundamental clinical studies, but access to the true initial rhythm has been a limiting factor.

METHODS: Using demographics and detailed clinical variables, we trained and tested an AI model (extreme gradient boosting) to differentiate PEA-SCA versus VF-SCA in a novel setting that provided the true initial rhythm. A subgroup of SCAs are witnessed by emergency medical services personnel, and because the response time is …

Out-Of-Distribution Detection In Long-Tailed Recognition With Calibrated Outlier Class Learning, Wenjun Miao, Guansong Pang, Xiao Bai, Tianqi Li, Jin Zheng

Research Collection School Of Computing and Information Systems

Existing out-of-distribution (OOD) methods have shown great success on balanced datasets but become ineffective in long-tailed recognition (LTR) scenarios where 1) OOD samples are often wrongly classified into head classes and/or 2) tail-class samples are treated as OOD samples. To address these issues, current studies fit a prior distribution of auxiliary/pseudo OOD data to the long-tailed in-distribution (ID) data. However, it is difficult to obtain such an accurate prior distribution given the unknowingness of real OOD samples and heavy class imbalance in LTR. A straightforward solution to avoid the requirement of this prior is to learn an outlier class to …