Artificial Intelligence and Robotics Commons^™

Genscore: Agent-Based Short-Answer Question Generation And Scoring In Software Engineering Courses, Nguyen Binh Duong Ta, Lwin Khin Shar

Research Collection School Of Computing and Information Systems

Short-answer questions are commonly used in educational assessments, as they are often viewed as a more effective way than multiple-choice questions to determine whether students have achieved the intended learning outcomes. However, manually creating appropriate questions targeting different cognitive levels such as those defined by the Bloom’s Taxonomy, and grading text answers from students are not trivial tasks for instructors. Existing work on auto-question generation and scoring in computing education typically targets coding-based questions. However, in software engineering courses, assessments can extend beyond coding to understanding of processes, DevOps methodologies, system design, etc. This work aims to address the dual …

Sempo: Lightweight Foundation Models For Time Series Forecasting, Hui He, Kun Yi, Yuanchi Ma, Qi Zhang, Zhengdong Niu, Guansong Pang

Research Collection School Of Computing and Information Systems

The recent boom of large pre-trained models witnesses remarkable success in developing foundation models (FMs) for time series forecasting. Despite impressive performance across diverse downstream forecasting tasks, existing time series FMs possess massive network architectures and require substantial pre-training on large-scale datasets, which significantly hinders their deployment in resource-constrained environments. In response to this growing tension between versatility and affordability, we propose SEMPO, a novel lightweight foundation model that requires pretraining on relatively small-scale data, yet exhibits strong general time series forecasting. Concretely, SEMPO comprises two key modules: 1) energy-aware SpEctral decomposition module, that substantially improves the utilization of pre-training …

Bias Testing And Mitigation In Llm-Based Code Generation, Dong Huang, Jie M. Zhang, Qingwen Bu, Xiaofei Xie, Junjie Chen, Heming Cui

Research Collection School Of Computing and Information Systems

As the adoption of LLMs becomes more widespread in software coding ecosystems, a pressing issue has emerged: does the generated code contain social bias and unfairness, such as those related to age, gender, and race? This issue concerns the integrity, fairness, and ethical foundation of software applications that depend on the code generated by these models but are underexplored in the literature. This paper presents a novel bias testing framework that is specifically designed for code generation tasks. Based on this framework, we conduct an extensive empirical study on the biases in code generated by five widely studied LLMs (i.e., …

Contx: Scene Context Prediction Via Context Bank And Layout Perception, Jingxin Liang, Yangyang Xu, Haorui Song, Yuan Lu, Yuhui Deng, Yiyi Long, Yan Huang, Shengxin Liu, Jianbo Jiao, Shengfeng He

Research Collection School Of Computing and Information Systems

Scene context prediction, which seeks to infer unknown contextual information from isolated object properties, currently faces limitations due to predominant reliance on pixel-wise supervision that overlooks real-world context priors. To address this, we present ContX, a context-prior-driven, coarse-to-fine model. ContX distinctively integrates explicit linguistic-contextual knowledge in two key ways. First, it proposes a linguistic guided context bank, leveraging linguistic-statistical contextual data to guide the rationality of segmentation shapes and foster meaningful inter-class contextual interactions. Second, ContX augments contextual comprehension by correlating layouts with linguistic descriptions, enhancing layout perception through a multi-modal strategy. Comprehensive experiments demonstrate ContX's superiority and versatility, outperforming …

Mando-Llm: Heterogeneous Graph Transformers With Large Language Models For Smart Contract Vulnerability Detection, Nhat Minh Nguyen, Huu Hoang Nguyen, Long Le Thanh, Zahra Ahmadi, Thanh Nam Doan, Daoyuan Wu, Lingxiao Jiang

Research Collection School Of Computing and Information Systems

Detecting vulnerabilities in smart contracts is vital for the security and reliability of decentralized apps. To facilitate vulnerability detection, contract codes, including bug patterns, are represented as heterogeneous graphs with various nodes and edges, like control-flow and function-call graphs. However, existing graph learning techniques struggle with large, complex graphs. This paper presents MANDO-LLM, a novel framework that combines heterogeneous graph transformers (HGTs) with large language models (LLMs) for detecting vulnerabilities in smart contracts represented as heterogeneous contract graphs built upon control-flow and call graphs. MANDO-LLM uses LLMs to capture code features from control-flow and call data, customizes HGTs to learn …

Agentguard: An Active Threat Discovery System For Package Confusion Using Multi-Agent Collaboration, Wei Ma, Yu Li, Zhi Chen, Ye Liu, Lingxiao Jiang, Qiang Hu, Junyi Tao

Research Collection School Of Computing and Information Systems

The proliferation of open-source software (OSS) has made software supply chains prime targets for attacks like Package Confusion, where adversaries publish malicious packages with names deceptively similar to legitimate ones. Existing detection methods often rely on simple lexical similarity or passive analysis of known package pairs, struggle with high false positive rates (FPR), fail to proactively identify emerging threats, and are vulnerable to adversarial evasion. To overcome these limitations, we introduce AgentGuard, a novel framework for proactive, single-input package confusion detection. AgentGuard employs a multi-agent architecture that autonomously discovers potential confusion targets using fine-tuned word embedding model to hybird semantic …

Introduction To Machine Learning And Machine Learning Systems, Raffi T. Khatchadourian Ph.D.

Open Educational Resources

Lecture slides introducing machine learning and machine learning systems for an undergraduate software engineering course. Topics include what machine learning is and how it differs from traditional programming, foundation models, the major types of learning (supervised, unsupervised, reinforcement, and others), and applications across domains. Using a food-delivery time-prediction case study, the deck walks through a typical ML pipeline—data collection and cleaning, feature engineering, model training, and evaluation—and covers evaluation methods (precision and recall, confusion matrices, error measures) along with underfitting versus overfitting and the realities of learning and evaluation in production. Based on "Machine Learning in Production/AI Engineering" by Christian …

Speculative Automated Refactoring Of Imperative Deep Learning Programs To Graph Execution, Raffi Khatchadourian Ph.D., Tatiana Castro Vélez, Mehdi Bagherzadeh, Nan Jia, Anita Raja

Publications and Research

Efficiency is essential to support ever-growing datasets, especially for Deep Learning (DL) systems. DL frameworks have traditionally embraced deferred execution-style DL code---supporting symbolic, graph-based Deep Neural Network (DNN) computation. While scalable, such development is error-prone, non-intuitive, and difficult to debug. Consequently, more natural, imperative DL frameworks encouraging eager execution have emerged but at the expense of run-time performance. Though hybrid approaches aim for the "best of both worlds," using them effectively requires subtle considerations. Our key insight is that, while DL programs typically execute sequentially, hybridizing imperative DL code resembles parallelizing sequential code in traditional systems. Inspired by this, we …

Speculative Automated Refactoring Of Imperative Deep Learning Programs To Graph Execution, Raffi Khatchadourian Ph.D., Tatiana Castro Vélez, Mehdi Bagherzadeh, Nan Jia, Anita Raja

Publications and Research

Efficiency is essential to support ever-growing datasets, especially for Deep Learning (DL) systems. DL frameworks have traditionally embraced deferred execution-style DL code---supporting symbolic, graph-based Deep Neural Network (DNN) computation. While scalable, such development is error-prone, non-intuitive, and difficult to debug. Consequently, more natural, imperative DL frameworks encouraging eager execution have emerged but at the expense of run-time performance. Though hybrid approaches aim for the "best of both worlds," using them effectively requires subtle considerations. Our key insight is that, while DL programs typically execute sequentially, hybridizing imperative DL code resembles parallelizing sequential code in traditional systems. Inspired by this, we …

Speculative Automated Refactoring Of Imperative Deep Learning Programs To Graph Execution, Raffi T. Khatchadourian Ph.D., Tatiana Castro Vélez, Mehdi Bagherzadeh, Nan Jia, Anita Raja

Publications and Research

Efficiency is essential to support ever-growing datasets, especially for Deep Learning (DL) systems. DL frameworks have traditionally embraced deferred execution-style DL code---supporting symbolic, graph-based Deep Neural Network (DNN) computation. While scalable, such development is error-prone, non-intuitive, and difficult to debug. Consequently, more natural, imperative DL frameworks encouraging eager execution have emerged but at the expense of run-time performance. Though hybrid approaches aim for the "best of both worlds," using them effectively requires subtle considerations. Our key insight is that, while DL programs typically execute sequentially, hybridizing imperative DL code resembles parallelizing sequential code in traditional systems. Inspired by this, we …

Designing For Novice Debuggers: A Pilot Study On An Ai-Assisted Debugging Tool, Oka Kurniawan, Erick Chandra, Christopher M. Poskitt, Yannic Noller, Kenny T.W. Choo, Cyrille Jegourel

Research Collection School Of Computing and Information Systems

Debugging is a fundamental skill that novice programmers must develop. Numerous tools have been created to assist novice programmers in this process. Recently, large language models (LLMs) have been integrated with automated program repair techniques to generate fixes for students' buggy code. However, many of these tools foster an over-reliance on AI and do not actively engage students in the debugging process. In this work, we aim to design an intuitive debugging assistant, CodeHinter, that combines traditional debugging tools with LLM-based techniques to help novice debuggers fix semantic errors while promoting active engagement in the debugging process. We present findings …

When Deep Learning Meets Information Retrieval-Based Bug Localization: A Survey, Feifei Niu, Chuanyi Li, Kui Liu, Xin Xia, David Lo

Research Collection School Of Computing and Information Systems

Bug localization is a crucial aspect of software maintenance, running through the entire software lifecycle. Information retrieval-based bug localization (IRBL) identifies buggy code based on bug reports, expediting the bug resolution process for developers. Recent years have witnessed significant achievements in IRBL, propelled by the widespread adoption of deep learning (DL). To provide a comprehensive overview of the current state of the art and delve into key issues, we conduct a survey encompassing 61 IRBL studies leveraging DL. We summarize best practices in each phase of the IRBL workflow, undertake a meta-analysis of prior studies, and suggest future research directions. …

Studying Satd In Drone Systems With Human-Ai Collaboration, Leevi Rantala, Lwin Khin Shar, Mäntylä Mika V., Wei Minn, Naing Tun Yan

Research Collection School Of Computing and Information Systems

Background: Self-Admitted Technical Debt (SATD) refers to sub-optimal solutions that developers acknowledge within the source code. SATD research originated on Java projects but is expanding to other domains. We focus on SATD in drones, which are used for various critical tasks.Aims: The primary objective is to investigate SATD in drone systems. The second aim is to explore the integration of AI and human collaboration for SATD labelling and classification.Method: We conducted a sample study of SATD comments in drone systems (14 open source, 4 SDKs) to analyse the quantity and types of SATD comments present. Our study incorporates collaboration between …

Human-Ai-Collaboration-For-Coding, Siddhardha Ravi

Theses, Dissertations and Culminating Projects

AI-generated code, while rapidly producing functional solutions, often falls short in aspects like comprehensive error handling, robust documentation, and optimal architectural design, areas where human expertise excels. Conversely, humans can greatly benefit from AI's rapid code generation capabilities. This project proposes and evaluates "A Framework to Improve Code Quality by Utilizing Generative AI Coding Along With Human-Written Code", designed to create a synergy between AI and human intelligence for enhanced software development. Conducted over four weeks, the research leverages BigCodeBench as its core dataset to rigorously investigate how human intervention can improve AI-generated code quality, identify the most effective human-AI …

Fact-Checker: A Web Application For Leveraging Large Language Models For Fact-Checking Youtube Videos, Andrew R. Craig

Electronic Theses, Projects, and Dissertations

Fact-Checker is a web application that allows users to fact-check YouTube videos. It feeds YouTube’s closed captioning transcript to a large language model (LLM) to extract claims. It then uses multiple LLMs, such as Gemini, Llama, and Claude, to verify these claims. The modular design makes it easy to change to a different LLM or model if needed. The application is built using Python for access to Application Programming Interfaces (APIs) and Streamlit as the front-end framework. The utilization of Docker and Dockerfiles enables easy distribution and deployment. It enables the application to be deployed on almost any hardware platform …

Freqllm: Frequency-Aware Large Language Models For Time Series Forecasting, Shunan Wang, Min Gao, Zongwei Wang, Yibing Bai, Feng Jiang, Guansong Pang

Research Collection School Of Computing and Information Systems

Large Language Models (LLMs) have recently shown promise in Time Series Forecasting (TSF) by effectively capturing intricate time-domain dependencies. However, our preliminary experiments reveal that standard LLM-based approaches often fail to capture global correlations, limiting predictive performance. We found that embedding frequency-domain signals smooths weight distributions and enhances structured correlations by clearly separating global trends (low-frequency components) from local variations (high-frequency components). Building on these insights, we propose FreqLLM, a novel framework that integrates frequency-domain semantic alignment into LLMs to refine prompts for improved time series analysis. By bridging the gap between frequency signals and textual embeddings, FreqLLM effectively captures …

Syntax-Enhanced Boundary-Aware Named Entity Recognition Model, Chuanming Yu, Bin Deng, Zhengang Zhang

Journal of Scientific Information Research

[Purpose/significance] This study addresses the issue of inadequate perception of entity boundaries in traditional character-level modeling-based named entity recognition models by integrating syntax information containing entity boundary features into the task using a multi-head graph attention network with dense connections. This integration enhances the effectiveness of named entity recognition.

[Method/process] This study proposes a Syntax-enhanced Boundary-aware Named Entity Recognition Model (SynBNER), which utilizes BERT for text semantic representation and integrates syntax information using a dense-connected graph attention network. This integration incorporates implicit entity boundary information from syntax information into word representations, thereby enhancing the model's entity boundary perception capability.

[Result/conclusion] …

Ecological Footprint Analysis Of Chatgpt (Gpt-3), Isabella Boulais

Computer Science and Software Engineering

Climate change is an escalating crisis that demands immediate action from all sectors, including the rapidly advancing field of artificial intelligence (AI). While AI offers climate solutions, its own environmental impact raises concerns. Unfortunately limited research due to rapid development, system complexity, and lack of standardized methodologies hinders our understanding of AI’s environmental consequences. This project aims to conduct a comprehensive ecological footprint analysis of OpenAI’s GPT-3 model that is used to power ChatGPT, establishing guidelines for assessing AI systems’ environmental impact and proposing a framework for improvement. Going beyond tracking carbon emissions, this project will outline the broader lifecycle …

Irhunter: Universal Detection Of Instruction Reordering Vulnerabilities For Enhanced Concurrency In Distributed And Parallel Systems, Guohua Xin, Guangquan Xu, Yao Zhang, Cheng Wen, Cen Zhang, Xiaofei Xie, Neal N. Xiong, Shaoying Liu, Pan Gao

Research Collection School Of Computing and Information Systems

Instruction reordering is an essential optimization technique used in both compilers and multi-core processors to enhance parallelism and resource utilization. Although the original intent of this technique is to benefit the program, some improper reordering can significantly impact the program correctness, which we call instruction reordering vulnerability (IRV). However, existing methods detect IRV by defining CPU instruction reordering rules to schedule execution paths while neglecting compiler reordering, and thus generate false positives that require manual filtering and resulting in inefficiency. To bridge this gap, in this paper, we propose the IRV detection method, , which analyzes IRV characteristics and extracts …

A Knowledge Enhanced Large Language Model For Bug Localization, Yue Li, Bohan Liu, Ting Zhang, Zhiqi Wang, David Lo, Lanxin Yang, Jun Lyu, He Zhang

Research Collection School Of Computing and Information Systems

A significant number of bug reports are generated every day as software systems continue to develop. Large Language Models (LLMs) have been used to correlate bug reports with source code to locate bugs automatically. The existing research has shown that LLMs are effective for bug localization and can increase software development efficiency. However, these studies still have two limitations. First, these models fail to capture context information about bug reports and source code. Second, these models are unable to understand the domain-specific expertise inherent to particular projects, such as version information in projects that are composed of alphanumeric characters without …

Demystifying Memorization In Llm-Based Program Repair Via A General Hypothesis Testing Framework, Jiaolong Kong, Xiaofei Xie, Shangqing Liu

Research Collection School Of Computing and Information Systems

Large Language Models (LLMs) have achieved remarkable success in various applications, particularly in code-related tasks such as code generation and program repair, setting new performance benchmarks. However, the extensive use of large training corpora raises concerns about whether these achievements stem from genuine understanding or mere memorization of training data—a question often overlooked in current research. This paper aims to study the memorization issue within LLM-based program repair by investigating whether the correct patches generated by LLMs are the result of memorization. The key challenge lies in the absence of ground truth for confirming memorization, leading to various ad-hoc methods …

Cashift: Benchmarking Log-Based Cloud Attack Detection Under Normality Shift, Jiongchi Yu, Xiaofei Xie, Qiang Hu, Bowen Zhang, Ziming Zhao, Yun Lin, Lei Ma, Ruitao Feng, Frank Liau

Research Collection School Of Computing and Information Systems

With the rapid advancement of cloud-native computing, securing cloud environments has become an important task. Log-based Anomaly Detection (LAD) is the most representative technique used in different systems for attack detection and safety guarantee, where multiple LAD methods and relevant datasets have been proposed. However, even though some of these datasets are specifically prepared for cloud systems, they only cover limited cloud behaviors and lack information from a whole-system perspective. Another critical issue to consider is normality shift, which implies that the test distribution could differ from the training distribution and highly affect the performance of LAD. Unfortunately, existing works …

Regtrieve: Reducing System-Level Regression Errors For Machine Learning Systems Via Retrieval-Enhanced Ensemble, Junming Cao, Xuwen Xiang, Mingfei Cheng, Bihuan Chen, Xinyan Wang, You Lu, Chaofeng Sha, Xiaofei Xie, Xin Peng

Research Collection School Of Computing and Information Systems

Multiple machine learning (ML) models are often incorporated into real-world ML systems. However, updating an individual model in these ML systems frequently results in regression errors, where the new model performs worse than the old model for some inputs. While model-level regression errors have been widely studied, little is known about how regression errors propagate at system level. To address this gap, we propose RegTrieve, a novel retrieval-enhanced ensemble approach to reduce regression errors at both model and system level. Our evaluation across various model update scenarios shows that RegTrieve reduces system-level regression errors with almost no impact on system …

Large Language Model For Vulnerability Detection And Repair: Literature Review And The Road Ahead, Xin Zhou, Sicong Cao, Xiaobing Sun, David Lo

Research Collection School Of Computing and Information Systems

The significant advancements in Large Language Models (LLMs) have resulted in their widespread adoption across various tasks within Software Engineering (SE), including vulnerability detection and repair. Numerous studies have investigated the application of LLMs to enhance vulnerability detection and repair tasks. Despite the increasing research interest, there is currently no existing survey that focuses on the utilization of LLMs for vulnerability detection and repair. In this paper, we aim to bridge this gap by offering a systematic literature review of approaches aimed at improving vulnerability detection and repair through the utilization of LLMs. The review encompasses research work from leading …

Evolving Enemy Behavior In Video Games, Hermie H. Adams Iii

Honors Theses

The video game I developed for my senior project lacked complex and engaging enemy artificial intelligence. The standard implementations of AI systems such as finite state machines and behavior trees felt like side-steps rather than innovative solutions. Upon seeing the 'magic' of machine learning in perfecting games such as Snake, Super Mario, and Flappy Bird, I was inspired to seek my answer in the field of evolutionary computation. However, my challenge differed in that the problem space would be defined by dynamic player strategies, making it not well-defined or static. As such, my evaluations are based on enemies exhibiting emergent …

Improving The Reproducibility Of Deep Learning Software: An Initial Investigation Through A Case Study Analysis, Nikita Ravi, Abhinav Goel, James C. Davis, George K. Thiruvathukal

Computer Science: Faculty Publications and Other Works

The field of deep learning has witnessed significant breakthroughs, spanning various applications, and fundamentally transforming current software capabilities. However, alongside these advancements, there have been increasing concerns about reproducing the results of these deep learning methods. This is significant because reproducibility is the foundation of reliability and validity in software development, particularly in the rapidly evolving domain of deep learning. The difficulty of reproducibility may arise due to several reasons, including having differences from the original execution environment, incompatible software libraries, proprietary data and source code, lack of transparency, and the stochastic nature in some software. A study conducted by …

Hybridize Functions: A Tool For Automatically Refactoring Imperative Deep Learning Programs To Graph Execution, Raffi Khatchadourian, Tatiana Castro Vélez, Mehdi Bagherzadeh, Nan Jia, Anita Raja

Publications and Research

Efficiency is essential to support responsiveness w.r.t. ever-growing datasets, especially for Deep Learning (DL) systems. DL frameworks have traditionally embraced deferred execution-style DL code—supporting symbolic, graph-based Deep Neural Network (DNN) computation. While scalable, such development is error-prone, non-intuitive, and difficult to debug. Consequently, more natural, imperative DL frameworks encouraging eager execution have emerged but at the expense of run-time performance. Though hybrid approaches aim for the “best of both worlds,” using them effectively requires subtle considerations to make code amenable to safe, accurate, and efficient graph execution—avoiding performance bottlenecks and semantically inequivalent results. We discuss the engineering aspects of a …

Hybridize Functions: A Tool For Automatically Refactoring Imperative Deep Learning Programs To Graph Execution, Raffi Khatchadourian, Tatiana Castro Vélez, Mehdi Bagherzadeh, Nan Jia, Anita Raja

Publications and Research

Efficiency is essential to support responsiveness w.r.t. ever-growing datasets, especially for Deep Learning (DL) systems. DL frameworks have traditionally embraced deferred execution-style DL code—supporting symbolic, graph-based Deep Neural Network (DNN) computation. While scalable, such development is error-prone, non-intuitive, and difficult to debug. Consequently, more natural, imperative DL frameworks encouraging eager execution have emerged but at the expense of run-time performance. Though hybrid approaches aim for the “best of both worlds,” using them effectively requires subtle considerations to make code amenable to safe, accurate, and efficient graph execution—avoiding performance bottlenecks and semantically inequivalent results. We discuss the engineering aspects of a …

Hybridize Functions: A Tool For Automatically Refactoring Imperative Deep Learning Programs To Graph Execution, Raffi Khatchadourian, Tatiana Castro Vélez, Mehdi Bagherzadeh, Nan Jia, Anita Raja

Publications and Research

Efficiency is essential to support responsiveness w.r.t. ever-growing datasets, especially for Deep Learning (DL) systems. DL frameworks have traditionally embraced deferred execution-style DL code—supporting symbolic, graph-based Deep Neural Network (DNN) computation. While scalable, such development is error-prone, non-intuitive, and difficult to debug. Consequently, more natural, imperative DL frameworks encouraging eager execution have emerged but at the expense of run-time performance. Though hybrid approaches aim for the "best of both worlds," using them effectively requires subtle considerations to make code amenable to safe, accurate, and efficient graph execution—avoiding performance bottlenecks and semantically inequivalent results. We discuss the engineering aspects of a …

Computer Vision In Soccer: Yolov11 Analytics Engine For Quantifying Game Strategy, Connor S. Maurer

Data Science Undergraduate Honors Theses

Single-shot object detection capabilities significantly reduce computational overhead for real-time computer vision in sports analytics at 60 FPS. YOLO11’s lightweight CNN gives promising accuracy while meeting the low-latency demand of dynamic soccer matches. As data-driven approaches take over the sport of soccer, efficient player tracking systems become critical for informing coach’s strategies. I prototype the ETL (Extract, Transform, Load) process of data collected from a single- shot detection program and evaluate its viability for estimating player fatigue. YOLO11 detects players, the ball, and other characteristics, with the output transformed by homography to estimate the positions in the real world. These …