Artificial Intelligence and Robotics Commons^™

Reducing Uncertainty In Sea-Level Rise Prediction: A Spatial-Variability-Aware Approach, Subhankar Ghosh, Shuai An, Arun Sharma, Jayant Gupta, Shashi Shekhar, Aneesh Subramanian

I-GUIDE Forum

Given multi-model ensemble climate projections, the goal is to accurately and reliably predict future sea-level rise while lowering the uncertainty. This problem is important because sea-level rise affects millions of people in coastal communities and beyond due to climate change's impacts on polar ice sheets and the ocean. This problem is challenging due to spatial variability and unknowns such as possible tipping points (e.g., collapse of Greenland or West Antarctic ice-shelf), climate feedback loops (e.g., clouds, permafrost thawing), future policy decisions, and human actions. Most existing climate modeling approaches use the same set of weights globally, during either regression or …

Chatgpt And Other Generative Ai: A Game Changer For Special Education?, Jennifer J. Lesh

Faculty and Staff Publications & Presentations

Keynote session that involved the audience, had them engage in share outs and exploring, as well as provided open-ended thought provoking questions to share with other stakeholders.

Peatmoss: Mining Pre-Trained Models In Open-Source Software, Wenxin Jiang, Jason Jones, Jerin Yasmin, Nicholas Synovic, Rajiv Sashti, Sophie Chen, George K. Thiruvathukal, Yuan Tian, James C. Davis

Computer Science: Faculty Publications and Other Works

Developing and training deep learning models is expensive, so software engineers have begun to reuse pre-trained deep learning models (PTMs) and fine-tune them for downstream tasks. Despite the widespread use of PTMs, we know little about the corresponding software engineering behaviors and challenges. To enable the study of software engineering with PTMs, we present the PeaTMOSS dataset: Pre-Trained Models in Open-Source Software. PeaTMOSS has three parts: a snapshot of (1) 281,638 PTMs, (2) 27,270 open-source software repositories that use PTMs, and (3) a mapping between PTMs and the projects that use them. We challenge PeaTMOSS miners to discover software engineering …

Synthesizing Sentience: Integrating Large Language Models And Autonomous Agents For Emulating Human Cognitive Complexity, Jay Ratican, James Hutson, Daniel Plate

Faculty Scholarship

The paper aims to present a novel methodology for emulating the intricacies of human cognitive complexity by ingeniously integrating large language models with autonomous agents. Grounded in the theoretical framework of the modular mind theory-originally espoused by Fodor and later refined by scholars such as Joanna Bryson—the study seeks to venture into the untapped potential of large language models and autonomous agents in mirroring human cognition. Recent advancements in artificial intelligence, exemplified by the inception of autonomous agents like Age in GPT, auto GPT, and baby AGI, underscore the transformative capacities of these technologies in diverse applications. Moreover, empirical studies …

Intelligence Versus Inferno: How Artificial Intelligence Can Be Used To Monitor And Manage Wildfires In Europe, Maxwell Feldman

Independent Study Project (ISP) Collection

Escalating wildfire occurrences in Europe, particularly in the southern Mediterranean region, are presenting significant challenges to socioeconomic, environmental, and ecosystem services. The increasing frequency and severity of these wildfires are straining resources and emphasizing the need for a better understanding of the relationship between suppression capacity and fire behavior in wildfire management. The following research addresses the urgent need for more proactive, knowledge-based, and technologically driven fire management strategies, throughout all four stages of wildfire response – fuel and land management, fire preparedness, fire suppression, and post-fire management. Artificial Intelligence (AI) is becoming increasingly significant in climate change adaptation, especially …

Csci 632/Csci 492: Machine Learning, David Harrison

GMAS Course Syllabi

No abstract provided.

An Ai-Based Framework For Translating American Sign Language To English And Vice Versa, Vijayendra D. Avina, Md Amiruzzaman, Stefanie Amiruzzaman, Linh B. Ngo, M. Ali Akber Dewan

Computer Science Faculty Publications

Abstract: In this paper, we propose a framework to convert American Sign Language (ASL) to English and English to ASL. Within this framework, we use a deep learning model along with the rolling average prediction that captures image frames from videos and classifies the signs from the image frames. The classified frames are then used to construct ASL words and sentences to support people with hearing impairments. We also use the same deep learning model to capture signs from the people with deaf symptoms and convert them into ASL words and English sentences. Based on this framework, we developed a …

Icl-D3ie: In-Context Learning With Diverse Demonstrations Updating For Document Information Extraction, Jiabang He, Lei Wang, Yi Hu, Ning Liu, Hui Liu, Xing Xu, Heng Tao Shen

Research Collection School Of Computing and Information Systems

arge language models (LLMs), such as GPT-3 and ChatGPT, have demonstrated remarkable results in various natural language processing (NLP) tasks with in-context learning, which involves inference based on a few demonstration examples. Despite their successes in NLP tasks, no investigation has been conducted to assess the ability of LLMs to perform document information extraction (DIE) using in-context learning. Applying LLMs to DIE poses two challenges: the modality and task gap. To this end, we propose a simple but effective in-context learning framework called ICL-D3IE, which enables LLMs to perform DIE with different types of demonstration examples. Specifically, we extract the …

Reachability Poorman Discrete-Bidding Games, Guy Avni, Tobias Meggendorfer, Suman Sadhukhan, Josef Tkadlec, Dorde Zikelic

Research Collection School Of Computing and Information Systems

We consider bidding games, a class of two-player zerosum graph games. The game proceeds as follows. Both players have bounded budgets. A token is placed on a vertex of a graph, in each turn the players simultaneously submit bids, and the higher bidder moves the token, where we break bidding ties in favor of Player 1. Player 1 wins the game iff the token visits a designated target vertex. Weconsider, for the first time, poorman discrete-bidding in which the granularity of the bids is restricted and the higher bid is paid to the bank. Previous work either did not impose …

Instance-Specific Algorithm Configuration Via Unsupervised Deep Graph Clustering, Wen Song, Yi Liu, Zhiguang Cao, Yaoxin Wu, Qiqiang Li

Research Collection School Of Computing and Information Systems

Instance-specific Algorithm Configuration (AC) methods are effective in automatically generating high-quality algorithm parameters for heterogeneous NP-hard problems from multiple sources. However, existing works rely on manually designed features to describe training instances, which are simple numerical attributes and cannot fully capture structural differences. Targeting at Mixed-Integer Programming (MIP) solvers, this paper proposes a novel instances-specific AC method based on end-to-end deep graph clustering. By representing an MIP instance as a bipartite graph, a random walk algorithm is designed to extract raw features with both numerical and structural information from the instance graph. Then an auto-encoder is designed to learn dense …

Voucher Abuse Detection With Prompt-Based Fine-Tuning On Graph Neural Networks, Zhihao Wen, Yuan Fang, Yihan Liu, Yang Guo, Shuji Hao

Research Collection School Of Computing and Information Systems

Voucher abuse detection is an important anomaly detection problem in E-commerce. While many GNN-based solutions have emerged, the supervised paradigm depends on a large quantity of labeled data. A popular alternative is to adopt self-supervised pre-training using label-free data, and further fine-tune on a downstream task with limited labels. Nevertheless, the "pre-train, fine-tune" paradigm is often plagued by the objective gap between pre-training and downstream tasks. Hence, we propose VPGNN, a prompt-based fine-tuning framework on GNNs for voucher abuse detection. We design a novel graph prompting function to reformulate the downstream task into a similar template as the pretext task …

Understanding The Effect Of Counterfactual Explanations On Trust And Reliance On Ai For Human-Ai Collaborative Clinical Decision Making, Min Hun Lee, Chong Jun Chew

Research Collection School Of Computing and Information Systems

Artificial intelligence (AI) is increasingly being considered to assist human decision-making in high-stake domains (e.g. health). However, researchers have discussed an issue that humans can over-rely on wrong suggestions of the AI model instead of achieving human AI complementary performance. In this work, we utilized salient feature explanations along with what-if, counterfactual explanations to make humans review AI suggestions more analytically to reduce overreliance on AI and explored the effect of these explanations on trust and reliance on AI during clinical decision-making. We conducted an experiment with seven therapists and ten laypersons on the task of assessing post-stroke survivors' quality …

Objectfusion: Multi-Modal 3d Object Detection With Object-Centric Fusion, Q. Cai, Y. Pan, T. Yao, Chong-Wah Ngo, T. Mei

Research Collection School Of Computing and Information Systems

Recent progress on multi-modal 3D object detection has featured BEV (Bird-Eye-View) based fusion, which effectively unifies both LiDAR point clouds and camera images in a shared BEV space. Nevertheless, it is not trivial to perform camera-to-BEV transformation due to the inherently ambiguous depth estimation of each pixel, resulting in spatial misalignment between these two multi-modal features. Moreover, such transformation also inevitably leads to projection distortion of camera image features in BEV space. In this paper, we propose a novel Object-centric Fusion (ObjectFusion) paradigm, which completely gets rid of camera-to-BEV transformation during fusion to align object-centric features across different modalities for …

Unsupervised Anomaly Detection In Medical Images With A Memory-Augmented Multi-Level Cross-Attentional Masked Autoencoder, Yu Tian, Guansong Pang, Yuyuan Liu, Chong Wang, Yuanhong Chen, Fengbei Liu, Rajvinder Singh, Johan W. Verjans, Mengyu Wang, Gustavo Carneiro

Research Collection School Of Computing and Information Systems

Unsupervised anomaly detection (UAD) aims to find anomalous images by optimising a detector using a training set that contains only normal images. UAD approaches can be based on reconstruction methods, self-supervised approaches, and Imagenet pre-trained models. Reconstruction methods, which detect anomalies from image reconstruction errors, are advantageous because they do not rely on the design of problem-specific pretext tasks needed by self-supervised approaches, and on the unreliable translation of models pre-trained from non-medical datasets. However, reconstruction methods may fail because they can have low reconstruction errors even for anomalous images. In this paper, we introduce a new reconstruction-based UAD approach …

Multi-Representation Variational Autoencoder Via Iterative Latent Attention And Implicit Differentiation, Nhu Thuat Tran, Hady Wirawan Lauw

Research Collection School Of Computing and Information Systems

Variational Autoencoder (VAE) offers a non-linear probabilistic modeling of user's preferences. While it has achieved remarkable performance at collaborative filtering, it typically samples a single vector for representing user's preferences, which may be insufficient to capture the user's diverse interests. Existing solutions extend VAE to model multiple interests of users by resorting a variant of self-attentive method, i.e., employing prototypes to group items into clusters, each capturing one topic of user's interests. Despite showing improvements, the current design could be more effective since prototypes are randomly initialized and shared across users, resulting in uninformative and non-personalized clusters.To fill the gap, …

Residual Pattern Learning For Pixel-Wise Out-Of-Distribution Detection In Semantic Segmentation, Y Liu, Choubo Ding, Yu Tian, Guansong Pang, Vasileios Belagiannis, Ian Reid, Gustavo Carneiro

Research Collection School Of Computing and Information Systems

Semantic segmentation models classify pixels into a set of known ("in-distribution") visual classes. When deployed in an open world, the reliability of these models depends on their ability to not only classify in-distribution pixels but also to detect out-of-distribution (OoD) pixels. Historically, the poor OoD detection performance of these models has motivated the design of methods based on model re-training using synthetic training images that include OoD visual objects. Although successful, these re-trained methods have two issues: 1) their in-distribution segmentation accuracy may drop during re-training, and 2) their OoD detection accuracy does not generalise well to new contexts (e.g., …

Feature Prediction Diffusion Model For Video Anomaly Detection, Cheng Yan, Shiyu Zhang, Yang Liu, Guansong Pang, Wenjun Wang

Research Collection School Of Computing and Information Systems

Anomaly detection in the video is an important research area and a challenging task in real applications. Due to the unavailability of large-scale annotated anomaly events, most existing video anomaly detection (VAD) methods focus on learning the distribution of normal samples to detect the substantially deviated samples as anomalies. To well learn the distribution of normal motion and appearance, many auxiliary networks are employed to extract foreground object or action information. These high-level semantic features effectively filter the noise from the background to decrease its influence on detection models. However, the capability of these extra semantic models heavily affects the …

Hallucination Detection: Robustly Discerning Reliable Answers In Large Language Models, Yuyuan Chen, Qiang Fu, Yichen Yuan, Zhihao Wen, Ge Fan, Dayiheng Liu, Dongmei Zhang, Zhixu Li, Yanghua Xiao

Research Collection School Of Computing and Information Systems

Large language models (LLMs) have gained widespread adoption in various natural language processing tasks, including question answering and dialogue systems. However, a major drawback of LLMs is the issue of hallucination, where they generate unfaithful or inconsistent content that deviates from the input source, leading to severe consequences. In this paper, we propose a robust discriminator named RelD to effectively detect hallucination in LLMs' generated answers. RelD is trained on the constructed RelQA, a bilingual question-answering dialogue dataset along with answers generated by LLMs and a comprehensive set of metrics. Our experimental results demonstrate that the proposed RelD successfully detects …

Dexbert: Effective, Task-Agnostic And Fine-Grained Representation Learning Of Android Bytecode, Tiezhu Sun, Kevin Allix, Kisub Kim, Xin Zhou, Dongsun Kim, David Lo, Tegawendé F. Bissyande, Jacques Klein

Research Collection School Of Computing and Information Systems

The automation of an increasingly large number of software engineering tasks is becoming possible thanks to Machine Learning (ML). One foundational building block in the application of ML to software artifacts is the representation of these artifacts ( e.g. , source code or executable code) into a form that is suitable for learning. Traditionally, researchers and practitioners have relied on manually selected features, based on expert knowledge, for the task at hand. Such knowledge is sometimes imprecise and generally incomplete. To overcome this limitation, many studies have leveraged representation learning, delegating to ML itself the job of automatically devising suitable …

Object Recognition With Deep Neural Networks In Low-End Systems, Lillian Davis

Mahurin Honors College Capstone Experience/Thesis Projects

Object recognition is an important area in computer vision. Object recognition has been advanced significantly by deep learning that unifies feature extraction and classification. In general, deep neural networks, such as Convolution Neural Networks (CNNs), are trained in high-performance systems. Aiming to extend the reach of deep learning to personal computing, I propose a study of deep learning-based object recognition in low-end systems, such as laptops. This research includes how differing layer configurations and hyperparameter values used in CNNs can either create or resolve the issue of overfitting and affect final accuracy levels of object recognition systems. The main contribution …

Improving Semantic Document Classification Accuracy By Integrating Human-Crafted Knowledge, Zachary Weinfeld, Lubomir Stanchev

College of Engineering Summer Undergraduate Research Program

Document classification is a pivotal task in various domains, warranting the development of robust algorithms. Among these, the Bidirectional Encoder Representations from Transformers (BERT) algorithm, introduced by Google, has proven to perform well when fine-tuned for the task at hand. Leveraging transformer architecture, BERT demonstrates stellar language understanding capabilities. However, the integration of BERT with a range of techniques has shown potential for further enhancing classification accuracy. This work investigates several techniques that leverage semantic understanding to improve the performance of document classification models trained with BERT. Specifically, we explore three methods. First, we will balance corpuses afflicted by imbalanced …

Machine Learning Prediction Of Hea Properties, Nicholas J. Beaver, Nathaniel Melisso, Travis Murphy

College of Engineering Summer Undergraduate Research Program

High-entropy alloys (HEA) are a very new development in the field of metallurgical materials. They are made up of multiple principle atoms unlike traditional alloys, which contributes to their high configurational entropy. The microstructure and properties of HEAs are are not well predicted with the models developed for more common engineering alloys, and there is not enough data available on HEAs to fully represent the complex behavior of these alloys. To that end, we explore how the use of machine learning models can be used to model the complex, high dimensional behavior in the HEA composition space. Based on our …

Exploring Approaches To Engage K-12 Students In Learning Computational Thinking Using Collaborative Robots, Zoila Anuri Kanu

College of Engineering Summer Undergraduate Research Program

Minority students are largely underrepresented in the STEM field. The goal for this project was to develop a program which promotes the inclusion of computation skills among students and help them work collaboratively with the use of human – robot interaction. Robots are such a strong tool that can be used to enhance computational thinking and engage students towards a technical field. Through workshops and readings about computational thinking we worked on building a block-based program that introduces the uses of robots as teaching tool for computational thinking.

Ethics And Social Justice For Ai In Data Science, Arya Ramchander, Kylene Nicole Landenberger

College of Engineering Summer Undergraduate Research Program

The advances of AI raise several critical questions about human values and ethics, highlighting the need for researchers and developers to consider the ethical implications and the risks of neglecting them. In the past few years, student researchers have developed an AI model that allows users to test their surveys for possible breaches of subject confidentiality. This allows the users to gauge the ethicality of their proposal. This summer, we have expanded on this research and launched an interactive model for students and researches to assess their current work for ethical and social justice implications. Using Langchain and Figma, we …

Ai For Search And Rescue - Locating A Missing Person, David Hernandez, Sai Rama Balakrishnan, Timmy Chin, Aditya Manikonda, Vasanth Pugalenthi

College of Engineering Summer Undergraduate Research Program

Building on the work done initially as a SURP 2021 project and continued through 2021-23, the focus for this summer project will be on the use of computer technology for locating a missing person. Over the last year, we developed the digital equivalents of about 30 paper-based S&R forms and the infrastructure to collect the respective information. In their current use, these paper forms are filled out by search teams, collected in a command post, and reviewed by search coordinators. This process is time-consuming, prone to errors and loss of information, and relies heavily on the experience, skills, and mental …

Integrating Human Expert Knowledge With Openai And Chatgpt: A Secure And Privacy-Enabled Knowledge Acquisition Approach, Ben Phillips

College of Engineering Summer Undergraduate Research Program

Advanced Large Language Models (LLMs) struggle to produce accurate results and preserve user privacy for use cases involving domain-specific knowledge. A privacy-preserving approach for leveraging LLM capabilities on domain-specific knowledge could greatly expand the use cases of LLMs in a variety of disciplines and industries. This project explores a method for acquiring domain-specific knowledge for use with GPT3 while protecting sensitive user information with ML-based text-sanitization.

A System For The Detection Of Adversarial Attacks In Computer Vision Via Performance Metrics, Sarah Reynolds

Doctoral Dissertations and Master's Theses

Adversarial attacks, or attacks committed by an adversary to hijack a system, are prevalent in the deep learning tasks of computer vision and are one of the greatest threats to these models' safe and accurate use. These attacks force the trained model to misclassify an image, using pixel-level changes undetectable to the human eye. Various defenses against these attacks exist and are detailed in this work. The work of previous researchers has established that when adversarial attacks occur, different node patterns in a Deep Neural Network (DNN) are activated within the model. Additionally, it is known that CPU and GPU …

Online Aircraft System Identification Using A Novel Parameter Informed Reinforcement Learning Method, Nathan Schaff

Doctoral Dissertations and Master's Theses

This thesis presents the development and analysis of a novel method for training reinforcement learning neural networks for online aircraft system identification of multiple similar linear systems, such as all fixed wing aircraft. This approach, termed Parameter Informed Reinforcement Learning (PIRL), dictates that reinforcement learning neural networks should be trained using input and output trajectory/history data as is convention; however, the PIRL method also includes any known and relevant aircraft parameters, such as airspeed, altitude, center of gravity location and/or others. Through this, the PIRL Agent is better suited to identify novel/test-set aircraft.

First, the PIRL method is applied to …

Spoken Language Processing And Modeling For Aviation Communications, Aaron Van De Brook

Doctoral Dissertations and Master's Theses

With recent advances in machine learning and deep learning technologies and the creation of larger aviation-specific corpora, applying natural language processing technologies, especially those based on transformer neural networks, to aviation communications is becoming increasingly feasible. Previous work has focused on machine learning applications to natural language processing, such as N-grams and word lattices. This thesis experiments with a process for pretraining transformer-based language models on aviation English corpora and compare the effectiveness and performance of language models transfer learned from pretrained checkpoints and those trained from their base weight initializations (trained from scratch). The results suggest that transformer language …

Application And Use Of Artificial Intelligence (Ai) For Library Services Delivery In Academic Libraries In Kwara State, Nigeria, Abdullahi Olayinka Isiaka

Library Philosophy and Practice (e-journal)

The application and use Artificial Intelligence (AI) in library services delivery and operations has modernized traditional practices, enabling libraries to adapt to the evolving information needs of patrons in the digital era. The main purpose of this study is to investigate the application and use of Artificial Intelligence (AI) Technologies for Library Services Delivery in Academic Libraries in Kwara State, Nigeria. The study used a descriptive survey approach. The population was the 108 librarians in academic libraries in Kwara State, Nigeria. A total enumeration technique was employed, and a questionnaire was used to collect data from the library staff. The …