Artificial Intelligence and Robotics Commons^™

Identifying Human Factor Causes Of Remotely Piloted Aircraft System Safety Occurrences In Australia, John Murray, Steven Richardson, Keith Joiner, Graham Wild

Research outputs 2022 to 2026

Remotely piloted aircraft are a fast-emerging sector of the aviation industry. Although technical failures have been the largest cause of accident occurrences for Remotely Piloted Aircraft Systems (RPASs), if they are to follow the path of conventionally crewed aviation, Human Factors (HFs) will increasingly contribute to accidents as the technology of RPASs improves. Examining an RPAS accident database from 2008–2019 for HF-caused accidents and coding to the Human Factors Analysis and Classification System (HFACS) taxonomy, an exploration of RPAS HFs is carried out and the predominant HF issues for RPAS pilots identified. The majority of HF accidents were coded to …

Pathways To Chronic Disease Detection And Prediction: Mapping The Potential Of Machine Learning To The Pathophysiological Processes While Navigating Ethical Challenges, Ebenezer Afrifa-Yamoah, Eric Adua, Emmanuel Peprah-Yamoah, Enoch O. Anto, Victor Opoku-Yamoah, Emmanuel Acheampong, Michael J. Macartney, Rashid Hashmi

Research outputs 2022 to 2026

Chronic diseases such as heart disease, cancer, and diabetes are leading drivers of mortality worldwide, underscoring the need for improved efforts around early detection and prediction. The pathophysiology and management of chronic diseases have benefitted from emerging fields in molecular biology like genomics, transcriptomics, proteomics, glycomics, and lipidomics. The complex biomarker and mechanistic data from these “omics” studies present analytical and interpretive challenges, especially for traditional statistical methods. Machine learning (ML) techniques offer considerable promise in unlocking new pathways for data-driven chronic disease risk assessment and prognosis. This review provides a comprehensive overview of state-of-the-art applications of ML algorithms for …

Quantifying The Role Of Active Listening And Reassurance In Virtual Health Coach Interactions, Ghulam Hussain, Brian Keegan, Robert Ross

Conference papers

Conversational Agents have the potential to support healthcare through coaching exercise routines, but are still lacking in demonstrating authentic social behaviours to support engagement. To this end, we present a series of experiments that we conducted in order to investigate how automated health care coaches can be more effective when their interaction style is tailored to demonstrate qualities associated with a good bedside manner, namely active listening and reassurance. To test this, we first developed a dataset of 135 dialogue excerpts from three distinct sources, i.e., original, handcrafted and LLMs, the latter two of which were tuned to demonstrate specific …

Patient Consent And The Right To Notice And Explanation Of Ai Systems Used In Health Care, Meghan E Hurley, Benjamin H Lang, Kristin Marie Kostick-Quenet, Jared N Smith, Jennifer Blumenthal-Barby

Center for Medical Ethics and Health Policy Staff Publications

Given the need for enforceable guardrails for artificial intelligence (AI) that protect the public and allow for innovation, the U.S. Government recently issued a Blueprint for an AI Bill of Rights which outlines five principles of safe AI design, use, and implementation. One in particular, the right to notice and explanation, requires accurately informing the public about the use of AI that impacts them in ways that are easy to understand. Yet, in the healthcare setting, it is unclear what goal the right to notice and explanation serves, and the moral importance of patient-level disclosure. We propose three normative functions …

Forward-Secure Hierarchical Delegable Signature For Smart Homes, Jianfei Sun, Guowen Xu, Yang Yang, Xuehuan Yang, Xiaoguo Li, Cong Wu, Zhen Liu, Guomin Yang, Robert H. Deng

Research Collection School Of Computing and Information Systems

Aiming to provide people with great convenience and comfort, smart home systems have been deployed in thousands of homes. In this paper, we focus on handling the security and privacy issues in such a promising system by customizing a new cryptographic primitive to provide the following security guarantees: (1) fine-grained, privacy-preserving authorization for smart home users and integrity protection of communication contents; (2) flexible self-sovereign permission delegation; (3) forward security of previous messages. To our knowledge, no previous system has been designed to consider these three security and privacy requirements simultaneously. To tackle these challenges, we put forward the first-ever …

Artificial Intelligence In Surgical Coding: Evaluating Large Language Models For Current Procedural Terminology Accuracy In Hand Surgery, Emily Isch, Jamie Lee, D. Mitchell Self, Abhijeet Sambangi, Theodore E. Habarth-Morales, John R. Vaile, E. J. Caterson

Department of Surgery Faculty Papers

PURPOSE: The advent of large language models (LLMs) like ChatGPT has introduced notable advancements in various surgical disciplines. These developments have led to an increased interest in the use of LLMs for Current Procedural Terminology (CPT) coding in surgery. With CPT coding being a complex and time-consuming process, often exacerbated by the scarcity of professional coders, there is a pressing need for innovative solutions to enhance coding efficiency and accuracy.

METHODS: This observational study evaluated the effectiveness of five publicly available large language models-Perplexity.AI, Bard, BingAI, ChatGPT 3.5, and ChatGPT 4.0-in accurately identifying CPT codes for hand surgery procedures. A …

Gender Biases Within Artificial Intelligence And Chatgpt: Evidence, Sources Of Biases, And Solutions, Qi Hui Jerlyn Ho, Andree Hartanto, Andrew Koh, Nadyanna M. Majeed

Research Collection School of Social Sciences

The growing adoption of Artificial Intelligence (AI) in various sectors has introduced significant benefits, but also raised concerns over biases, particularly in relation to gender. Despite AI's potential to enhance sectors like healthcare, education, and business, it often mirrors reality and its societal prejudices and can manifest itself through unequal treatment in hiring decisions, academic recommendations, or healthcare diagnostics, systematically disadvantaging women. This paper explores how AI systems and chatbots, notably ChatGPT, can perpetuate gender biases due to inherent flaws in training data, algorithms, and user feedback loops. This problem stems from several sources, including biased training datasets, algorithmic design …

Divide-And-Conquer: Confluent Triple-Flow Network For Rgb-T Salient Object Detection, Hao Tang, Zechao Li, Dong Zhang, Shengfeng He, Jinhui Tang

Research Collection School Of Computing and Information Systems

RGB-Thermal Salient Object Detection (RGB-T SOD) aims to pinpoint prominent objects within aligned pairs of visible and thermal infrared images. A key challenge lies in bridging the inherent disparities between RGB and Thermal modalities for effective saliency map prediction. Traditional encoder-decoder architectures, while designed for cross-modality feature interactions, may not have adequately considered the robustness against noise originating from defective modalities, thereby leading to suboptimal performance in complex scenarios. Inspired by hierarchical human visual systems, we propose the ConTriNet, a robust Confluent Triple-Flow Network employing a "Divide-and-Conquer"strategy. This framework utilizes a unified encoder with specialized decoders, each addressing different subtasks …

Simulation-Free Hierarchical Latent Policy Planning For Proactive Dialogues, Tao He, Lizi Liao, Yixin Cao, Yuanxing Liu, Yiheng Sun, Zerui Chen, Ming Liu, Bing Qin

Research Collection School Of Computing and Information Systems

Recent advancements in proactive dialogues have garnered significant attention, particularly for more complex objectives (e.g. emotion support and persuasion). Unlike traditional task-oriented dialogues, proactive dialogues demand advanced policy planning and adaptability, requiring rich scenarios and comprehensive policy repositories to develop such systems. However, existing approaches tend to rely on Large Language Models (LLMs) for user simulation and online learning, leading to biases that diverge from realistic scenarios and result in suboptimal efficiency. Moreover, these methods depend on manually defined, context-independent, coarse-grained policies, which not only incur high expert costs but also raise concerns regarding their completeness. In our work, we …

Dr. Tongue: Sign-Oriented Multi-Label Detection For Remote Tongue Diagnosis, Yiliang Chen, Steven S. C. Ho, Cheng Xu, Yao Jie Xie, Wing Fai Yeung, Shengfeng He, Jing Qin

Research Collection School Of Computing and Information Systems

Tongue diagnosis is a vital tool in Western and Traditional Chinese Medicine, providing key insights into a patient's health by analyzing tongue attributes. The COVID-19 pandemic has heightened the need for accurate remote medical assessments, emphasizing the importance of precise tongue attribute recognition via telehealth. To address this, we propose a Sign-Oriented multi-label Attributes Detection framework. Our approach begins with an adaptive tongue feature extraction module that standardizes tongue images and mitigates environmental factors. This is followed by a Sign-oriented Network (SignNet) that identifies specific tongue attributes, emulating the diagnostic process of experienced practitioners and enabling comprehensive health evaluations. To …

Token Turing Machines Are Efficient Vision Models, Purvish Jajal, Nick Eliopoulos, Benjamin Shiue-Hal Chou, George K. Thiruvathukal, James C. Davis, Yung-Hsiang Lu

Computer Science: Faculty Publications and Other Works

We propose Vision Token Turing Machines (ViTTM), an efficient, low-latency, memory-augmented Vision Transformer (ViT). Our approach builds on Neural Turing Machines (NTM) and Token Turing Machines (TTM), which were applied to NLP and sequential visual understanding tasks. ViTTMs are designed for non-sequential computer vision tasks such as image classification and segmentation. Our model creates two sets of tokens: process tokens and memory tokens; process tokens pass through encoder blocks and read-write from memory tokens at each encoder block in the network, allowing them to store and retrieve information from memory. By ensuring that there are fewer process tokens, we are …

Respear: Earable-Based Robust Respiratory Rate Monitoring, Yang Liu, Kayla-Jade Butkow, Jake Stuchbury-Wass, Adam Pullin, Dong Ma, Cecilia Masolo

Research Collection School Of Computing and Information Systems

Respiratory rate (RR) monitoring is integral to understanding physical and mental health and tracking fitness. Existing studies have demonstrated the feasibility of RR monitoring under specific user conditions (e.g., while remaining still, or while breathing heavily). Yet, performing accurate, continuous and non-obtrusive RR monitoring across diverse daily routines and activities remains challenging. In this work, we present RespEar, an earable-based system for robust RR monitoring. By leveraging the unique properties of in-ear microphones in earbuds, RespEar enables the use of Respiratory Sinus Arrhythmia (RSA) and Locomotor Respiratory Coupling (LRC), physiological couplings between cardiovascular activity, gait and respiration, to indirectly determine …

Imageinthat: Manipulating Images To Convey User Instructions To Robots, Karthik Mahadevan, Blaine Lewis, Jiannan Li, Bilge Mutlu, Anthony Tang, Tovi Grossman

Research Collection School Of Computing and Information Systems

Foundation models are rapidly improving the capability of robots in performing everyday tasks autonomously such as meal preparation, yet robots will still need to be instructed by humans due to model performance, the difficulty of capturing user preferences, and the need for user agency. Robots can be instructed using various methods---natural language conveys immediate instructions but can be abstract or ambiguous, whereas end-user programming supports longer-horizon tasks but interfaces face difficulties in capturing user intent. In this work, we propose using direct manipulation of images as an alternative paradigm to instruct robots, and introduce a specific instantiation called ImageInThat which …

Adaptive Deviation Learning For Visual Anomaly Detection With Data Contamination, Aanindya Sundar Das, Guansong Pang, Monowar Bhuyan

Research Collection School Of Computing and Information Systems

Visual anomaly detection targets to detect images that notably differ from normal pattern, and it has found extensive application in identifying defective parts within the manufacturing industry. These anomaly detection paradigms predominantly focus on training detection models using only clean, unlabeled normal samples, assuming an absence of contamination; a condition often unmet in real-world scenarios. The performance of these methods significantly depends on the quality of the data and usually decreases when exposed to noise. We introduce a systematic adaptive method that employs deviation learning to compute anomaly scores end-to-end while addressing data contamination by assigning relative importance to the …

Generalization Analysis For Deep Contrastive Representation Learning, Minh Hieu Nong, Antoine Ledent, Yunwen Lei, Cheng Yeaw Ku

Research Collection School Of Computing and Information Systems

In this paper, we present generalization bounds for the unsupervised risk in the Deep Contrastive Representation Learning framework, which employs deep neural networks as representation functions. We approach this problem from two angles. On the one hand, we derive a parameter-counting bound that scales with the overall size of the neural networks. On the other hand, we provide a norm-based bound that scales with the norms of neural networks’ weight matrices. Ignoring logarithmic factors, the bounds are independent of k, the size of the tuples provided for contrastive learning. To the best of our knowledge, this property is only shared …

Explainable Neural Networks With Guarantees: A Sparse Estimation Approach, Antoine Ledent, Peng Liu

Research Collection School Of Computing and Information Systems

Balancing predictive power and interpretability has long been a challenging research area, particularly in powerful yet complex models like neural networks, where nonlinearity obstructs direct interpretation. This paper introduces a novel approach to constructing an explainable neural network that harmonizes predictiveness and explainability. Our model is designed as a linear combination of a sparse set of jointly learned features, each derived from a different trainable function applied to a single 1-dimensional input feature. Leveraging the ability to learn arbitrarily complex relationships, our neural network architecture enables automatic selection of a sparse set of important features, with the final prediction being …

Revisiting Sentiment Analysis For Software Engineering In The Era Of Large Language Models, Ting Zhang, Ivana Clairine Irsan, Thung Ferdian, David Lo

Research Collection School Of Computing and Information Systems

Software development involves collaborative interactions where stakeholders express opinions across various platforms. Recognizing the sentiments conveyed in these interactions is crucial for the effective development and ongoing maintenance of software systems. For software products, analyzing the sentiment of user feedback, e.g., reviews, comments, and forum posts can provide valuable insights into user satisfaction and areas for improvement. This can guide the development of future updates and features. However, accurately identifying sentiments in software engineering datasets remains challenging.This study investigates bigger large language models (bLLMs) in addressing the labeled data shortage that hampers fine-tuned smaller large language models (sLLMs) in software …

Multi-Uav Reconnaissance Mission Planning Via Deep Reinforcement Learning With Simulated Annealing, Mingfeng Fan, Huan Liu, Guohua Wu, Aldy Gunawan, Guillaume Sartoretti

Research Collection School Of Computing and Information Systems

Unmanned aerial vehicles (UAVs) are widely used in reconnaissance missions due to their autonomy and flexibility. Efficient mission planning for multiple UAVs is crucial for tasks such as traffic monitoring and data collection. However, existing approaches to multi-UAV reconnaissance mission planning problem (MURMPP) often struggle with high computational demands, leading to suboptimal solutions. To overcome this challenge, we introduce a divide-and-conquer framework that splits the problem into two phases: target allocation and UAV routing, effectively reducing computational complexity. Specifically, we propose a hybrid method, SA-NNO-DRL, which combines the nearest neighbor optima-based deep reinforcement learning (NNO-DRL) approach with simulated annealing (SA). …

Multisfl: Towards Accurate Split Federated Learning Via Multi-Model Aggregation And Knowledge Replay, Zeke Xia, Ming Hu, Dengke Yan, Ruixuan Liu, Anran Li, Xiaofei Xie, Mingsong Chen

Research Collection School Of Computing and Information Systems

Although Split Federated Learning (SFL) effectively enables knowledge sharing among resource-constrained clients, it suffers from low training performance due to the neglect of data heterogeneity and catastrophic forgetting problems. To address these issues, we propose a novel SFL approach named MultiSFL, which adopts i) an effective multimodel aggregation mechanism to alleviate gradient divergence caused by heterogeneous data and ii) a novel knowledge replay strategy to deal with the catastrophic forgetting problem. MultiSFL adopts two servers (i.e., the fed server and main server) to maintain multiple branch models for local training and an aggregated master model for knowledge sharing among branch …

Ragg: Retrieval-Augmented Grasp Generation Model, Zhenhua Tang, Bin Zhu, Yanbin Hao, Chong-Wah Ngo, Richang Hong

Research Collection School Of Computing and Information Systems

Intent-based grasp generation inherently involves challenges such as manipulation ambiguity and modality gaps. To address these, we propose a novel Retrieval-Augmented Grasp Generation model (RAGG). Our key insight is that when humans manipulate new objects, they initially mimic the interaction patterns observed in similar objects, then progressively adjust hand-object contact. Consequently, we develop RAGG as a two-stage approach, encompassing retrieval-guided generation and structurally stable grasp refinement. In the first stage, we propose a Retrieval-Augmented Diffusion Model (ReDim), which identifies the most relevant interaction instance from a knowledge base to explicitly guide grasp generation, thereby mitigating ambiguity and bridging modality gaps …