Artificial Intelligence and Robotics Commons^™

Llms-Based Augmentation For Domain Adaptation In Long-Tailed Food Datasets, Qing Wang, Chong-Wah Ngo, Ee-Peng Lim, Qianru Sun

Research Collection School Of Computing and Information Systems

Training a model for food recognition is challenging because the training samples, which are typically crawled from the Internet, are visually different from the pictures captured by users in the free-living environment. In addition to this domain-shift problem, the real-world food datasets tend to be long-tailed distributed and some dishes of different categories exhibit subtle variations that are difficult to distinguish visually. In this paper, we present a framework empowered with large language models (LLMs) to address these challenges in food recognition. We first leverage LLMs to parse food images to generate food titles and ingredients. Then, we project the …

Interactive Video Search With Multi-Modal Llm Video Captioning, Yu-Tong Cheng, Jiaxin Wu, Zhixin Ma, Jiangshan He, Xiao-Yong Wei, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

Cross-modal representation learning is essential for interactive text-to-video search tasks. However, the representation learning is limited by the size and quality of video-caption pairs. To improve the search accuracy, we propose to enlarge the size of available video-caption pairs by leveraging multi-model LLM on video captioning. Specifically, we use LLM to generate video captions for a large video collection (i.e., WebVid dataset) and use the generated video-caption pairs to pre-train a text-to-video search model. Additionally, we use LLM to generate fine-grained captions for test video collections to enable text-to-caption retrieval. Furthermore, we build a semantic overview of the retrieved rank …

Weakly-Supervised Semantic Segmentation With Image-Level Labels: From Traditional Models To Foundation Models, Zhaozheng Chen, Qianru Sun

Research Collection School Of Computing and Information Systems

The rapid development of deep learning has driven significant progress in image semantic segmentation—a fundamental task in computer vision. Semantic segmentation algorithms often depend on the availability of pixel-level labels (i.e., masks of objects), which are expensive, time consuming, and labor intensive. Weakly supervised semantic segmentation (WSSS) is an effective solution to avoid such labeling. It utilizes only partial or incomplete annotations and provides a cost-effective alternative to fully supervised semantic segmentation. In this article, our focus is on the WSSS with image-level labels, which is the most challenging form of WSSS. Our work has two parts. First, we conduct …

Synthesizing Multi-Person And Rare Pose Images For Human Pose Estimation, Liuqing Zhao, Zichen Tian, Zou Peng, Richang Hong, Qianru Sun

Research Collection School Of Computing and Information Systems

Human pose estimation (HPE) models underperform in recognizing rare poses because they suffer from data imbalance problems (i.e., there are few image samples for rare poses) in their training datasets. From a data perspective, the most intuitive solution is to synthesize data for rare poses. Specifically, the rule-based methods apply manual manipulations (such as Cutout and GridMask) to the existing data, so the limited diversity of the data constrains the model. An alternative method is to learn the underlying data distribution via deep generative models (such as ControlNet and HumanSD) and then sample “new data” from the distribution. This works …

Transforming Urban Dynamics: Harnessing Large Language Models For Smarter Mobility, Hao Xue, Ming Jin, Shirui Pan, Flora Salim, Guansong Pang

Research Collection School Of Computing and Information Systems

Artificial intelligence (AI) has the potential to analyze mobility data and make mobility systems smarter by leveraging diverse data sources such as geospatial data, transportation logs, and real-time sensor data to optimize traffic flow, enhance public transportation systems, and support the development of autonomous vehicles. With the newly emerged generative AI paradigm, exemplified by large language models (LLMs), there is great potential to transform the current AI applications in mobility, transportation, and urban domains. This article provides an overview of recent efforts and aims to shed light on the challenges and future opportunities to facilitate the adaptation of LLMs for …

A Survey Of Multilingual Large Language Models, Libo Qin, Qiguang Chen, Yuhang Zhou, Zhi Chen, Yinghui Li, Lizi Liao, Min Li, Wanxiang Che, Philip S. Yu

Research Collection School Of Computing and Information Systems

Multilingual large language models (MLLMs) leverage advanced large language models to process and respond to queries across multiple languages, achieving significant success in polyglot tasks. Despite these breakthroughs, a comprehensive survey summarizing existing approaches and recent developments remains absent. To this end, this paper presents a unified and thorough review of the field, highlighting recent progress and emerging trends in MLLM research. The contributions of this paper are as follows. (1) Extensive survey: to our knowledge, this is the pioneering thorough review of multilingual alignment in MLLMs. (2) Unified taxonomy: we provide a unified framework to summarize the current progress …

Adversarial Generative Flow Network For Solving Vehicle Routing Problems, Ni Zhang, Jingfeng Yang, Zhiguang Cao, Xu Chi

Research Collection School Of Computing and Information Systems

Recent research into solving vehicle routing problems (VRPs) has gained significant traction, particularly through the application of deep (reinforcement) learning for end-to-end solution construction. However, many current construction-based neural solvers predominantly utilize Transformer architectures, which can face scalability challenges and struggle to produce diverse solutions. To address these limitations, we introduce a novel framework beyond Transformer-based approaches, i.e., Adversarial Generative Flow Networks (AGFN). This framework integrates the generative flow network (GFlowNet)-a probabilistic model inherently adept at generating diverse solutions (routes)-with a complementary model for discriminating (or evaluating) the solutions. These models are trained alternately in an adversarial manner to improve …

The Effectiveness Of Local Updates For Decentralized Learning Under Data Heterogeneity, Tongle Wu, Zhize Li, Ying Sun

Research Collection School Of Computing and Information Systems

We revisit two fundamental decentralized optimization methods, Decentralized Gradient Tracking (DGT) and Decentralized Gradient Descent (DGD), with multiple local updates. We consider two settings and demonstrate that incorporating local update steps can reduce communication complexity. Specifically, for  $\mu$-strongly convex and $L$-smooth loss functions, we proved that local DGT  achieves communication complexity {}{$\tilde{\mathcal{O}} \Big(\frac{L}{\mu(K+1)} + \frac{\delta + {}{\mu}}{\mu (1 - \rho)} + \frac{\rho }{(1 - \rho)^2} \cdot \frac{L+ \delta}{\mu}\Big)$}, where $K$ is the number of additional local update}, $\rho$ measures the network connectivity and $\delta$ measures the second-order heterogeneity of the local losses. Our results reveal the tradeoff between communication and …

Defending Federated Recommender Systems Against Untargeted Attacks: A Contribution-Aware Robust Aggregation Scheme, Ruicheng Liang, Yuanchun Jiang, Feida Zhu, Ling Cheng, Huiwen Liu

Research Collection School Of Computing and Information Systems

Federated recommender systems (FedRSs) effectively tackle the tradeoff between recommendation accuracy and privacy preservation. However, recent studies have revealed severe vulnerabilities in FedRSs, particularly against untargeted attacks seeking to undermine their overall performance. Defense methods employed in traditional recommender systems are not applicable to FedRSs, and existing robust aggregation schemes for other federated learning-based applications have proven ineffective in FedRSs. Building on the observation that malicious clients contribute negatively to the training process, we design a novel contribution-aware robust aggregation scheme to defend FedRSs against untargeted attacks, named contribution-aware Bayesian knowledge distillation aggregation (ConDA), comprising two key components for the …

Fedart: A Neural Model Integrating Federated Learning And Adaptive Resonance Theory, Shubham Pateria, Budhitama Subagdja, Ah-Hwee Tan

Research Collection School Of Computing and Information Systems

Federated Learning (FL) has emerged as a promising paradigm for collaborative model training across distributed clients while preserving data privacy. However, prevailing FL approaches aggregate the clients’ local models into a global model through multi-round iterative parameter averaging. This leads to the undesirable bias of the aggregated model towards certain clients in the presence of heterogeneous data distributions among the clients. Moreover, such approaches are restricted to supervised classification tasks and do not support unsupervised clustering. To address these limitations, we propose a novel one-shot FL approach called Federated Adaptive Resonance Theory (FedART) which leverages self-organizing Adaptive Resonance Theory (ART) …

Deep Reinforcement Learning With Explicit Context Representation, Francisco Munguia-Galeano, Ah-Hwee Tan, Ze Ji

Research Collection School Of Computing and Information Systems

Though reinforcement learning (RL) has shown an outstanding capability for solving complex computational problems, most RL algorithms lack an explicit method that would allow learning from contextual information. On the other hand, humans often use context to identify patterns and relations among elements in the environment, along with how to avoid making wrong actions. However, what may seem like an obviously wrong decision from a human perspective could take hundreds of steps for an RL agent to learn to avoid. This article proposes a framework for discrete environments called Iota explicit context representation (IECR). The framework involves representing each state …

Large Scale Machine Learning Over Knowledge Graphs, Bedirhan Gergin

Electronic Theses & Dissertations (2024 - present)

Knowledge graphs (KGs) have become popular across various fields, providing convenient access to web-based knowledge while storing and formalizing domain-specific information. By analyzing KGs, patterns, connections, and dependencies can be identified across different data sources, enabling the inference of new knowledge from given facts. As the use of KGs expands, the size of modern KGs has grown significantly, making them impossible to process within the main memory of a single computer. Distributed computing offers a viable solution to this challenge by leveraging the combined capabilities of multiple servers within a cluster. This thesis explores how distributed computing can be effectively …

Improving Generalizability In Image Manipulation Detection, Zhenfei Zhang

Electronic Theses & Dissertations (2024 - present)

Image manipulation detection (IMD) aims to determine whether an image has been tampered with and to identify the manipulated regions. These capabilities have become increasingly important with the rapid advancement of media editing and generation technologies, such as Photoshop and generative AI methods, which underscore the need for robust tools for media authentication. Although current state-of-the-art (SoTA) methods achieve strong results on common manipulation types, such as splicing, copy-move, and removal, they often struggle to generalize to manipulation types not represented in the training data. Consequently, their real-world applicability remains limited, with performance degrading significantly in practical scenarios.

In this …

Flexible Hybrid Self-Powered Piezo-Triboelectric Nanogenerator Based On Bto-Pvdf/Pdms Nanocomposites For Human Machine Interaction, Wentao Dong, Mengyun Li, Chang Chen, Kun Xie, Jinhua Hong, Lin Yang

Civil & Environmental Engineering Faculty Publications

As flexible and wearable electronics play more and more important role in smart watches, smart glass and virtual reality, and the power supply to the wearable electronics have been revealed more attentions for long-term usage and continuous healthy monitoring. To overcome the challenge, flexible self-powered BTO-PVDF/PDMS piezoelectric-triboelectric electric hybrid generators (BPP-HNG) are developed to human gesture monitoring and human machine interaction (HMI) application without external power supply. BPP-HNG based on BTO-PVDF and PDMS films are prepared by sol-gel and spin-coating method. When the BTO content is 20 wt.%, BPP-HNG exhibits better electrical performance with an output voltage of 20.51 V. …

Pushing The Boundaries Of Large Language Models: Innovations And Limitations In Nlp, Finance, And Mathematics, A M Muntasir Rahman

Dissertations

Large Language Models (LLMs) have emerged as transformative tools across a spectrum of domains, yet their practical deployment reveals a blend of remarkable potential and notable limitations. This research explores innovative methodologies to extend the capabilities of LLMs while addressing critical challenges in their evaluation and application. By leveraging rule-based approaches, the in-context learning capabilities of LLMs, and human-in-the-loop validation across three focused studies, this research introduces robust strategies for dataset synthesis, model enhancement, and model assessment in three distinct domains: natural language processing, financial sentiment analysis, and mathematical reasoning

The first study proposes an efficient data augmentation framework, EASE, …

Surveying The Role Of Visual Analytics In Human-Machine Teaming, Naga Datha Saikiran Battula

Theses

Humans and machines both possess their unique capabilities and have their strengths and weaknesses, which can be complementary to one another and allow them to achieve a common goal. Teaming in the modern era involves text prompts, voice commands, gesture recognition, touch interfaces, and the latest visualization techniques that allow parties/agents to interact. Communication through visualization plays a vital role in allowing robust insights to be gained through a glance. Using visualization as a medium between humans and machines can increase the communication bandwidth. Human-machine teaming has witnessed much progress, with many theories and practical examples emerging. In the report, …

Automated Segmentation Of The Ulnar Nerve In Mri Using Deep Learning Techniques, Akhil Nagulapalli

Theses

Cubital Tunnel Syndrome (CuTS), a condition caused by compression of the ulnar nerve, results in numbness, tingling, pain, and even muscle atrophy, affecting fine motor skills and diminishing patient quality of life. Accurate diagnosis of CuTS is challenging, as current diagnostic methods—including clinical exams, nerve conduction studies, and unaided MRI—often lack the precision to reliably identify the nerve and detect compression in its early stages. Deep learning-based segmentation offers a promising solution, enabling precise and automated identification of nerve structures in MRI images, which could significantly improve diagnostic accuracy and support timely intervention.

A novel deep learning model for segmenting …

Ai-Assisted Academia: Unveiling Doctoral Students' Perspectives On Dissertation In Practice Innovation, Jennifer J. Lesh, Jévaughn J. Lancaster

Faculty and Staff Publications & Presentations

This action research study explores 73 doctoral students' perceptions of using Generative Artificial Intelligence (GAI) throughout their research journey in one educational doctorate (Ed.D) program. The first phase employed surveys, while the second incorporated semi-structured focus group interviews based on the survey data from a diverse sample of students across educational disciplines currently enrolled in the university's educational leadership doctoral program. In the study's first phase, the survey quantified educators' familiarity with, attitudes towards, perceived challenges, ethical considerations, and benefits of using GAI in doctoral research. The exploration of GAI in this practitioner-inspired doctoral program has uncovered essential insights into …

Cropsync: Ai-Powered Sustainable Crop Management, Ziad Doughan, Ibrahim Mneimneh, Zouheir Nakouzi, Noor Al Khaib, Samer Damaj, Jamal Chaaban, Hamza Mrad, Sari Itani

BAU Journal - Science and Technology

CropSync is a smart agriculture system that uses AI and IoT technologies to enable sustain- able crop management and precision farming. The system aims to address the challenges faced by the agriculture sector, such as increasing food production to meet global population demands while minimizing environmental impact. CropSync integrates sensors, cameras, and cloud-based analytics to provide farmers with real-time insights and recommendations for optimizing crop cul- tivation. The system upholds engineering professional and ethical standards, considering broader social, environmental, and economic implications. From a social perspective, CropSync improves food security and enhances farmers’ livelihoods through increased productivity and efficient re- …

Artificial Intelligence In Fetal And Pediatric Echocardiography, Alan Wang, Tam T Doan, Charitha Reddy, Pei-Ni Jone

Faculty, Staff and Students Publications

Echocardiography is the main modality in diagnosing acquired and congenital heart disease (CHD) in fetal and pediatric patients. However, operator variability, complex image interpretation, and lack of experienced sonographers and cardiologists in certain regions are the main limitations existing in fetal and pediatric echocardiography. Advances in artificial intelligence (AI), including machine learning (ML) and deep learning (DL), offer significant potential to overcome these challenges by automating image acquisition, image segmentation, CHD detection, and measurements. Despite these promising advancements, challenges such as small number of datasets, algorithm transparency, physician comfort with AI, and accessibility must be addressed to fully integrate AI …

Measurement Of Breast Artery Calcification Using An Artificial Intelligence Detection Model And Its Association With Major Adverse Cardiovascular Events, Suzanne Rose, Josette Hartnett, Zachary Estep, Daniyal Ameen, Shweta Karki, Edward Schuster, Rebecca Newman, David Hsi

Department of Medicine Faculty Papers

Breast artery calcification (BAC) obtained from standard mammographic images is currently under evaluation to stratify risk of major adverse cardiovascular events in women. Measuring BAC using artificial intelligence (AI) technology, we aimed to determine the relationship between BAC and coronary artery calcification (CAC) severity with Major Adverse Cardiac Events (MACE). This retrospective study included women who underwent chest computed tomography (CT) within one year of mammography. T-test assessed the associations between MACE and variables of interest (BAC versus MACE, CAC versus MACE). Risk differences were calculated to capture the difference in observed risk and reference groups. Chi-square tests and/or Fisher's …

Editorial: Artificial Intelligence For Smart Health: Learning, Simulation, And Optimization, Bing Yao, Nathan Gaw, Hyo Kyung Lee

Faculty Publications

With rapid developments in medical sensing and imaging, we now live in an era of data explosion in which large amounts of data are readily available in clinical environments. The fast-growing biomedical and healthcare data provide unprecedented opportunities for data-driven scientific knowledge discovery and clinical decision support. Our Research Topic aims to catalyze synergies among biomedical informatics, machine learning, computer simulation, operations research, systems engineering, and other related fields with three specific goals: (1) develop cutting-edge data-driven models to accelerate scientific knowledge discovery in biomedicine using healthcare data collected from laboratory systems, imaging systems, and medical and sensing devices; (2) …

Ethical Aspects Of Utilising Artificial Intelligence In Clinical Settings, Jeffrey Byrnes, Michael Robinson

Philosophy Faculty Articles and Research

In response to recent proposals to utilize artificial intelligence (AI) to automate ethics consultations in healthcare, we raise two main problems for the prospect of having healthcare professionals rely on AI-driven programs to provide ethical guidance in clinical matters. The first cause for concern is that, because these programs would effectively function like black boxes, this approach seems to preclude the kind of transparency that would allow clinical staff to explain and justify treatment decisions to patients, fellow caregivers, and those tasked with providing oversight. The other main problem is that the kind of authority that would need to be …

Transparency And Authority Concerns With Using Ai To Make Ethical Recommendations In Clinical Settings, Jeffrey Byrnes, Michael Robinson

Philosophy Faculty Articles and Research

In response to recent proposals to utilize artificial intelligence (AI) to automate ethics consultations in healthcare, we raise two main problems for the prospect of having healthcare professionals rely on AI-driven programs to provide ethical guidance in clinical matters. The first cause for concern is that, because these programs would effectively function like black boxes, this approach seems to preclude the kind of transparency that would allow clinical staff to explain and justify treatment decisions to patients, fellow caregivers, and those tasked with providing oversight. The other main problem is that the kind of authority that would need to be …

Simulation Of Cascade Failure In Urban Rail Transit Hypernetworks Based On Hypergraph Theory, Zijin Han, Mingjun Qian, Xixian Wang, Kaiyue Zhang

Journal of System Simulation

Abstract: In order to enhance the resilience of urban rail transit networks to ensure stable operations and passenger safety in the face of emergencies, hypergraph theory is introduced to construct a hypergraph based urban rail transit hypernetwork model, and a nonlinear load-capacity cascading failure model based on passenger flow weighting is established. In response to the passenger evacuation process at actual transportation network stations, a load redistribution mechanism is proposed, taking into consideration both the network level and the importance of passenger flow. To address scenarios where stations in actual traffic networks can still accommodate loads during shutdowns, a node …

A Fast Federated Learning-Based Crypto-Aggregation Scheme And Its Simulation Analysis, Boshen Lü, Xiao Song

Journal of System Simulation

Abstract: To solve the problem of increased computation and communication costs caused by using homomorphic encryption (HE) to protect all gradients in traditional cryptographic aggregation (cryptoaggregation) schemes, a fast crypto-aggregation scheme called RandomCrypt was proposed. RandomCrypt performed clipping and quantization to fix the range of gradient values and then added two types of noise on the gradient for encryption and differential privacy (DP) protection. It conducted HE on noise keys to revise the precision loss caused by DP protection. RandomCrypt was implemented based on a FATE framework, and a hacking simulation experiment was conducted. The results show that the proposed …

Critical Node Identification Method For Unmanned Aerial Vehicle Cluster Considering Localized Features, Chenglong Shi, Xiang Hua, Dong Wang, Jinjin Zhang, Tianqi Jiang, Yuanzhang Dang

Journal of System Simulation

Abstract: Aiming at the problem that the UAV cluster critical node identification methods focus on the global network and ignore the correlation between nodes and their local features, a critical nodes identification method for unmanned aerial vehicle cluster considering local features is proposed. An unmanned aerial vehicle cluster network model is constructed based on complex network theory. The Laplacian energy is introduced to evaluate the importance of node within two hops, and information entropy is combined to evaluate the importance of node in a specific motif to comprehensive identify the critical nodes. Simulation results demonstrate that this method identifies critical …

Optimization Of Urban Agglomeration Transportation Network Evacuation Paths, Bowei He, Chengbing Li, Shida Nie, Jialin Wang

Journal of System Simulation

Abstract: Given the complexity of the internal transportation network structure within urban agglomerations and the presence of numerous alternative routes, this paper proposes an enhanced ant colony algorithm to address the evacuation path problem of urban agglomeration transportation networks. A comprehensive urban agglomeration transportation network model is constructed, in which the issue of virtual transfer edges within the urban scope is considered and a weighting function is constructed taking into account the travelling time cost and the transferring time cost. Optimizations are applied to the ant colony algorithm, constructing an adaptive adjustment of state transitions and an information pheromone update …

A Threat Assessment Method In Uncertain Dynamic Environments, Mei Yang, Bingkun Wang, Zhongjie Zhang, Yan Zeng, Jian Huang

Journal of System Simulation

Abstract: A threat assessment method based on priori information and dynamic observation results is studied for the existence of dynamic uncertainty in complex war systems. The data mining is applied to obtain prior knowledge on the battlefield situation and construct an equipment-related confidence matrix. The sensor model is constructed to dynamically update the number of blue-side entities under the current situation by using the Bayesian method and considering both intelligence and observation results. The threat evaluation indicators and their weights are determined, and the TOPSIS method is used to finish the threat assessment. This method can well describe the complex …

Research On Adaptive Scheduling Of Single-Arm Cluster Tools For Throughput Ratio Of Multiple Wafer Types With Concurrent Processing, Chunrong Pan, Yu Cui, Wenqing Xiong, Hao Zhou, Jiliang Luo

Journal of System Simulation

Abstract: Concurrent processing makes the wafer fabrication process prone to deadlocks and completion node ambiguity. A Petri net model is established to describe the system operation process by taking for the single-arm cluster tools for fully parallel processing of two wafer types as the research object, and a control strategy is developed to avoid the system deadlock. Based on the Petri net model, the temporal properties of the system is analyzed based on earliest starting strategy, and the action cycle sequence of robot is determined during the monitoring cycle for different scenarios of lot switching in a single production monitoring …