Computer Sciences Commons^™

Backdoorllm: A Comprehensive Benchmark For Backdoor Attacks And Defenses On Large Language Models, Yige Li, Hanxun Huang, Yunhan Zhao, Xingjun Ma, Jun Sun

Research Collection School Of Computing and Information Systems

Generative large language models (LLMs) have achieved state-of-the-art results on a wide range of tasks, yet they remain susceptible to backdoor attacks: carefully crafted triggers in the input can manipulate the model to produce adversaryspecified outputs. While prior research has predominantly focused on backdoor risks in vision and classification settings, the vulnerability of LLMs in open-ended text generation remains underexplored. To fill this gap, we introduce BackdoorLLM1 , the first comprehensive benchmark for systematically evaluating backdoor threats in text-generation LLMs. BackdoorLLM provides: (i) a unified repository of benchmarks with a standardized training and evaluation pipeline; (ii) a diverse suite of …

A Comparative Evaluation Of Feedback Strategies For Enhancing Student Software Test Suite Writing Outcomes, Ashton Alonge

Master's Theses

Background and Context

Software testing is a fundamental component of computer science education, forming the basis for students’ ability to ensure program correctness and reliability. Despite its importance, many students struggle to design test cases that effectively expose faults and achieve meaningful test coverage. Traditional instructional approaches often emphasize code coverage metrics such as line or branch coverage, but these metrics may not adequately capture the quality of student tests. Mutation analysis, which measures how well tests detect small, artificial faults (mutants) introduced into the program, offers a potentially richer measure of test effectiveness. However, little is known about how …

Introduction To Machine Learning And Machine Learning Systems, Raffi T. Khatchadourian Ph.D.

Open Educational Resources

Lecture slides introducing machine learning and machine learning systems for an undergraduate software engineering course. Topics include what machine learning is and how it differs from traditional programming, foundation models, the major types of learning (supervised, unsupervised, reinforcement, and others), and applications across domains. Using a food-delivery time-prediction case study, the deck walks through a typical ML pipeline—data collection and cleaning, feature engineering, model training, and evaluation—and covers evaluation methods (precision and recall, confusion matrices, error measures) along with underfitting versus overfitting and the realities of learning and evaluation in production. Based on "Machine Learning in Production/AI Engineering" by Christian …

Speculative Automated Refactoring Of Imperative Deep Learning Programs To Graph Execution, Raffi Khatchadourian Ph.D., Tatiana Castro Vélez, Mehdi Bagherzadeh, Nan Jia, Anita Raja

Publications and Research

Efficiency is essential to support ever-growing datasets, especially for Deep Learning (DL) systems. DL frameworks have traditionally embraced deferred execution-style DL code---supporting symbolic, graph-based Deep Neural Network (DNN) computation. While scalable, such development is error-prone, non-intuitive, and difficult to debug. Consequently, more natural, imperative DL frameworks encouraging eager execution have emerged but at the expense of run-time performance. Though hybrid approaches aim for the "best of both worlds," using them effectively requires subtle considerations. Our key insight is that, while DL programs typically execute sequentially, hybridizing imperative DL code resembles parallelizing sequential code in traditional systems. Inspired by this, we …

Speculative Automated Refactoring Of Imperative Deep Learning Programs To Graph Execution, Raffi Khatchadourian Ph.D., Tatiana Castro Vélez, Mehdi Bagherzadeh, Nan Jia, Anita Raja

Publications and Research

Efficiency is essential to support ever-growing datasets, especially for Deep Learning (DL) systems. DL frameworks have traditionally embraced deferred execution-style DL code---supporting symbolic, graph-based Deep Neural Network (DNN) computation. While scalable, such development is error-prone, non-intuitive, and difficult to debug. Consequently, more natural, imperative DL frameworks encouraging eager execution have emerged but at the expense of run-time performance. Though hybrid approaches aim for the "best of both worlds," using them effectively requires subtle considerations. Our key insight is that, while DL programs typically execute sequentially, hybridizing imperative DL code resembles parallelizing sequential code in traditional systems. Inspired by this, we …

Speculative Automated Refactoring Of Imperative Deep Learning Programs To Graph Execution, Raffi T. Khatchadourian Ph.D., Tatiana Castro Vélez, Mehdi Bagherzadeh, Nan Jia, Anita Raja

Publications and Research

Efficiency is essential to support ever-growing datasets, especially for Deep Learning (DL) systems. DL frameworks have traditionally embraced deferred execution-style DL code---supporting symbolic, graph-based Deep Neural Network (DNN) computation. While scalable, such development is error-prone, non-intuitive, and difficult to debug. Consequently, more natural, imperative DL frameworks encouraging eager execution have emerged but at the expense of run-time performance. Though hybrid approaches aim for the "best of both worlds," using them effectively requires subtle considerations. Our key insight is that, while DL programs typically execute sequentially, hybridizing imperative DL code resembles parallelizing sequential code in traditional systems. Inspired by this, we …

Effects Of Code Scaffolding In Increasing Student Confidence In Programming Cryptography, John Denny

LSU Master's Theses

Cryptography is essential for secure communications, and new threats require more students willing to program and interact with cryptographic systems. Previous research is focused on tools for teaching these systems at a high level, teaching through attacks against these systems, and proper use of these systems in software development. In this paper, we seek to design a workshop to use scaffolded Python code to teach how these cryp- tographic systems are designed. We explore the use of code scaffolding for students to program an example implementation of the McEliece crypto- graphic system to build confidence in working with these systems. …

Trustworthy Federated Learning Framework For Secure, Efficient, And Quality-Aware Distributed Ai, Asadullah Tariq

Dissertations

Federated Learning (FL) emerged as a significant advancement in the field of Artificial Intelligence (AI), enabling collaborative model training across distributed devices while maintaining data privacy. As the importance of FL and its application in various areas increased, addressing trustworthiness issues in its various aspects became crucial. In the FL process, clients contribute updates computed on their local datasets, which the server aggregates to iteratively refine the global model. However, not all client data may be relevant to the learning objective, and incorporating updates from irrelevant data can harm the model's performance. The selection of training samples significantly impacts model …

Sketch-Sparsenet: Sparse Convolution Framework For Sketch Recognition, Jingru Yang, Jin Wang, Yang Zhou, Guodong Lu, Yu Sun, Huan Yu, Heming Fang, Zhihui Li, Shengfeng He

Research Collection School Of Computing and Information Systems

In free-hand sketch recognition, state-of-the-art methods often struggle to extract spatial features from sketches with sparse distributions, which are characterized by significant blank regions devoid of informative content. To address this challenge, we introduce a novel framework for sketch recognition, termed Sketch-SparseNet. This framework incorporates an advanced convolutional component: the Sketch-Driven Dilated Deformable Block (SD3B). This component excels at extracting spatial features and accurately recognizing free-hand sketches with sparse distributions. The SD3B component innovatively bridges gaps in the blank areas of sketches by establishing spatial relationships among disconnected stroke points through adaptive reshaping of convolution kernels. These kernels are deformable, …

When Deep Learning Meets Information Retrieval-Based Bug Localization: A Survey, Feifei Niu, Chuanyi Li, Kui Liu, Xin Xia, David Lo

Research Collection School Of Computing and Information Systems

Bug localization is a crucial aspect of software maintenance, running through the entire software lifecycle. Information retrieval-based bug localization (IRBL) identifies buggy code based on bug reports, expediting the bug resolution process for developers. Recent years have witnessed significant achievements in IRBL, propelled by the widespread adoption of deep learning (DL). To provide a comprehensive overview of the current state of the art and delve into key issues, we conduct a survey encompassing 61 IRBL studies leveraging DL. We summarize best practices in each phase of the IRBL workflow, undertake a meta-analysis of prior studies, and suggest future research directions. …

The Impact Of Sanctions On Github Developers And Activities, Youmei Fan, Ani Hovhannisyan, Hideaki Hata, Christoph Treude, Raula G. Kula

Research Collection School Of Computing and Information Systems

The GitHub platform has fueled the creation of truly global software, enabling contributions from developers across various geographical regions of the world. As software becomes more entwined with global politics and social regulations, it becomes similarly subject to government sanctions. In 2019, GitHub restricted access to certain services for users in specific locations but rolled back these restrictions for some communities (e.g., the Iranian community) in 2021. We conducted a largescale empirical study, collecting approximately 156 thousand user profiles and their 41 million activity points from 2008 to 2022, to understand the response of developers. Our results indicate that many …

Generative Ai And Empirical Software Engineering: A Paradigm Shift, Christoph Treude, Margaret-Anne Storey

Research Collection School Of Computing and Information Systems

The widespread adoption of generative AI in software engineering marks a paradigm shift, offering new opportunities to design and utilize software engineering tools while influencing both developers and the artifacts they create. Traditional empirical methods in software engineering, including quantitative, qualitative, and mixed-method approaches, are well established. However, this paradigm shift introduces novel data types and redefines many concepts in the software engineering process. The roles of developers, users, agents, and researchers increasingly overlap, blurring the distinctions between these social and technical actors within the field. This paper examines how integrating AI into software engineering challenges traditional research paradigms. It …

Enhancing Spatial Understanding In Mixed-Reality Presentations, Nam-Dang Vo, Van-Vinh Thai, Nam-Hoi Do, Viet-Tham Huynh, Anthony Tang, Khan-Duy Le

Research Collection School Of Computing and Information Systems

Mixed reality (MR) presentations often involve a presenter wearing a head-mounted display (HMD) and an audience watching via a large display, making it difficult for audiences to perceive spatial relationships between the presenter and virtual objects. We report two experiments testing three design variations: (1) scene camera placement (audience-aligned vs. opposite), (2) overlaying the presenter’s first-person view, and (3) highlighting objects in the presenter’s view. Results show that audience-aligned cameras and object highlighting improve spatial understanding, while combining third- and first-person views can further aid perception. We derive design guidelines for configuring MR presentations to better support audience comprehension.

Symbolic Execution Engine For Dynamic Analysis Of System Software, Pansilu Madhura Bhashana Pitigala Arachchillage

Dissertations and Theses Collection (Open Access)

System software, like any regular software, is prone to errors. It plays a specific role in a computer system by managing the underlying hardware and providing a platform to execute the application software. Defective or vulnerable system software can be exploited by attackers to compromise the entire system. Therefore, the system software must be studied and thoroughly analyzed to evaluate its security. However, due to the inherent complexity and its close interactions with the hardware, analyzing system software is a challenging task. As a result, there is a lack of tools and techniques capable of effectively analyzing system software.

This …

Designing For Novice Debuggers: A Pilot Study On An Ai-Assisted Debugging Tool, Oka Kurniawan, Erick Chandra, Christopher M. Poskitt, Yannic Noller, Kenny T.W. Choo, Cyrille Jegourel

Research Collection School Of Computing and Information Systems

Debugging is a fundamental skill that novice programmers must develop. Numerous tools have been created to assist novice programmers in this process. Recently, large language models (LLMs) have been integrated with automated program repair techniques to generate fixes for students' buggy code. However, many of these tools foster an over-reliance on AI and do not actively engage students in the debugging process. In this work, we aim to design an intuitive debugging assistant, CodeHinter, that combines traditional debugging tools with LLM-based techniques to help novice debuggers fix semantic errors while promoting active engagement in the debugging process. We present findings …

Simulated Interactive Debugging, Yannic Noller, Erick Chandra, Srinidhi Chandrashekar, Kenny Choo, Cyrille Jegourel, Oka Kurniawan, Christopher M. Poskitt

Research Collection School Of Computing and Information Systems

Debugging software, i.e., the localization of faults and their repair, is a key activity in software engineering. Therefore, effective and efficient debugging is one of the core skills a software engineer must develop. However, the teaching of debugging techniques is usually very limited or only taught in indirect ways, e.g., during software projects. As a result, most Computer Science (CS) students learn debugging only in an ad-hoc and unstructured way. In this work, we present our approach called Simulated Interactive Debugging that interactively guides students along the debugging process. The guidance aims to empower the students to repair their solutions …

Interaction2code: Benchmarking Mllm-Based Interactive Webpage Code Generation From Interactive Prototyping, Jingyu Xiao, Yuxuan Wan, Yintong Huo, Zixin Wang, Xinyi Xu, Wenxuan Wang, Zhiyao Xu, Yuhang Wang, Michael R. Lyu

Research Collection School Of Computing and Information Systems

Multimodal Large Language Models (MLLMs) have demonstrated remarkable performance on the design-to-code task, i.e., generating UI code from UI mock-ups. However, existing benchmarks only contain static web pages for evaluation and ignore the dynamic interaction, limiting the practicality, usability and user engagement of the generated webpages. To bridge these gaps, we present the first systematic investigation of MLLMs in generating interactive webpages. Specifically, we formulate the Interaction-to-Code task and establish the Interaction2Code benchmark, encompassing 127 unique webpages and 374 distinct interactions across 15 webpage types and 31 interaction categories. Through comprehensive experiments utilizing state-of-theart (SOTA) MLLMs, evaluated via both automatic …

Sustainable Llm Inference For Edge Ai: Evaluating Quantized Llms For Energy Efficiency, Output Accuracy, And Inference Latency, Erik Johanne Husom, Arda Goknil, Merve Astekin, Lwin Khin Shar, Andre Kasen, Sagar Sen, Benedikt Andreas Mithassel, Ahmet Soylu

Research Collection School Of Computing and Information Systems

Deploying Large Language Models (LLMs) on edge devices presents significant challenges due to computational constraints, memory limitations, inference speed, and energy consumption. Model quantization has emerged as a key technique to enable efficient LLM inference by reducing model size and computational overhead. In this study, we conduct a comprehensive analysis of 28 quantized LLMs from the Ollama library, which applies by default Post-Training Quantization (PTQ) and weight-only quantization techniques, deployed on an edge device (Raspberry Pi 4 with 4GB RAM). We evaluate energy efficiency, inference performance, and output accuracy across multiple quantization levels and task types. Models are benchmarked on …

Envisioning Future Interactive Web Development: Editing Webpage With Natural Language, Truong Hai Dang, Jingyu Xiao, Yintong Huo

Research Collection School Of Computing and Information Systems

The evolution of web applications relies on iterative code modifications, a process that is traditionally manual and time-consuming. While Large Language Models (LLMs) can generate UI code, their ability to edit existing code from new design requirements (e.g., ”center the logo”) remains a challenge. This is largely due to the absence of large-scale, high-quality tuning data to align model performance with human expectations. In this paper, we introduce a novel, automated data generation pipeline that uses LLMs to synthesize a high-quality fine-tuning dataset for web editing, named Instruct4Edit. Our approach generates diverse instructions, applies the corresponding code modifications, and performs …

Exploring Autonomous Agents: A Closer Look At Why They Fail When Completing Tasks, Ruofan Lu, Yichen Li, Yintong Huo

Research Collection School Of Computing and Information Systems

Autonomous agent systems powered by Large Language Models (LLMs) have demonstrated promising capabilities in automating complex tasks. However, current evaluations largely rely on success rates without systematically analyzing the interactions, communication mechanisms, and failure causes within these systems. To bridge this gap, we present a benchmark of 34 representative programmable tasks designed to rigorously assess autonomous agents. Using this benchmark, we evaluate three popular open-source agent frameworks combined with two LLM backbones, observing a task completion rate of approximately 50%. Through in-depth failure analysis, we develop a three-tier taxonomy of failure causes aligned with task phases, highlighting planning errors, task …

Teaching Diffusion Models To Ground Alpha Matte, Tianyi Xiang, Weiying Zheng, Yutao Jiang, Tingrui Shen, Hewei Yu, Yangyang Xu, Shengfeng He

Research Collection School Of Computing and Information Systems

The power of visual language models is showcased in visual understanding tasks, where language-guided models achieve impressive flexibility and precision. In this paper, we ex tend this capability to the challenging domain of image matting by framing it as a soft grounding problem, enabling a single diffusion model to handle diverse objects, textures, and transparencies, all directed by descriptive text prompts. Our method teaches the diffusion model to ground alpha mattes by guiding it through a process of instance-level localization and transparency estimation. First, we introduce an intermediate objective that trains the model to accurately localize semantic components of the …

Enhancing Llm Code Generation: A Systematic Evaluation Of Multi-Agent Collaboration And Runtime Debugging For Improving Accuracy, Reliability, And Latency, Nazmus Ashrafi

Theses

The use of large language models (LLMs) for automated code generation has emerged as a significant focus within AI research. As these pretrained models continue to evolve, their ability to understand and generate complex code structures has opened up new possibilities for automating intricate programming tasks with greater accuracy. Although contemporary foundational models demonstrate promising results, researchers continue to explore optimal post-training strategies to enhance code quality. These include supervised fine-tuning, retrieval-augmented generation (RAG), debugging, and many others. In this thesis, I combine two such widely used post training approaches—namely (1) multi agent collaboration and (2) runtime execution of information-based …

Reproducibility Debt In Scientific Software, Zara Hassan, Christoph Treude, Graham Williams, Michael Norrish, Alex Potanin

Research Collection School Of Computing and Information Systems

Reproducibility Debt (RpD) refers to accumulated technical and organisational issues in scientific software that hinder the ability to reproduce research results. While reproducibility is essential to scientific integrity, RpD remains poorly defined and under-addressed. This study introduces a formal definition of RpD and investigates its causes, effects, and mitigation strategies using a mixed-methods approach involving a systematic literature review (214 papers), interviews (23 practitioners), and a global survey (59 participants). We identify seven categories of contributing issues, 75 causes, 110 effects, and 61 mitigation strategies. Findings are synthesised into a cause-effect model and supported by taxonomies of team roles and …

Viewsrd: 3d Visual Grounding Via Structured Multi-View Decomposition, Ronggang Huang, Haoxin Yang, Yan Cai, Xuemiao Xu, Huaidong Zhang, Shengfeng He

Research Collection School Of Computing and Information Systems

3Dvisual grounding aims to identify and localize objects in a 3Dspacebasedontextualdescriptions. However, existing methods struggle with disentangling targets from anchors in complex multi-anchor queries and resolving inconsisten cies in spatial descriptions caused by perspective variations. To tackle these challenges, we propose ViewSRD, a frame work that formulates 3D visual grounding as a structured multi-view decomposition process. First, the Simple Rela tion Decoupling (SRD) module restructures complex multi anchor queries into a set of targeted single-anchor state ments, generating a structured set of perspective-aware de scriptions that clarify positional relationships. These de composed representations serve as the foundation for the Multi-view …

Emoshortcuts: Emotionally Expressive Body Augmentation For Social Mixed Reality Avatars, Hyuna Seo, Youngki Lee, Rajesh Krishna Balan, Thivya Kandappu

Research Collection School Of Computing and Information Systems

We present EmoShortcuts1, a novel social Mixed Reality (MR) framework that enhances emotional expression by dynamically augmenting avatar body gestures to reflect users’ emotional states. While social MR enables immersive remote interactions through avatars, conveying emotions remains challenging due to limitations in head-mounted display (HMD) tracking (e.g., missing lower-body movements, such as stomping or defensive postures), and users’ tendency to deprioritize nonverbal expressions during multitasking. EmoShortcuts addresses these challenges by introducing an augmentation framework that generates expressive body gestures even when users’ physical movements are restricted. We conducted a formative study with 12 participants to identify key challenges in emotional …

Tactile Data Comics: Combining Step-By-Step Presentation Of Tactile Graphics With Verbal Narration For The Blind And Visually Impaired, Yang Jiao, Ruoting Sun, Rong Luo, Xiwen Yao, Xinran She, Kotaro Hara, Yuewen Zhang, Xinyi Fu

Research Collection School Of Computing and Information Systems

Tactile graphics on a refreshable display have proven effective in enabling visually impaired people to comprehend pictorial content. To further evaluate the effectiveness of refreshable tactile displays in blind education, we designed tactile data comics, a method that combines step-by-step presentation of tactile graphics with verbal narration. We conducted a user study with sixteen visually impaired students to compare tactile data comics against verbal-only and static tactile graphics. Our findings show that tactile data comics significantly improve participants’ comprehension and engagement during the learning experience. These empirical results suggest that the integration of refreshable tactile displays and tactile data comics …

Information-Bottleneck Driven Binary Neural Network For Change Detection, Kaijie Yin, Zhiyuan Zhang, Shu Kong, Tian Gao, Cheng-Zhong Xu, Hui Kong

Research Collection School Of Computing and Information Systems

In this paper, we propose Binarized Change Detection (BiCD), the first binary neural network (BNN) designed specifically for change detection. Conventional network binarization approaches, which directly quantize both weights and activations in change detection models, severely limit the network's ability to represent input data and distinguish between changed and unchanged regions. This results in significantly lower detection accuracy compared to real-valued networks. To overcome these challenges, BiCD enhances both the representational power and feature separability of BNNs, improving detection performance. Specifically, we introduce an auxiliary objective based on the Information Bottleneck (IB) principle, guiding the encoder to retain essential input …

Morphology-Aware Hrv Estimation From Wrist Ppg In Sedentary Scenarios, Changshuo Hu, Hung Manh Pham, Dong Ma

Research Collection School Of Computing and Information Systems

Photoplethysmography (PPG) is widely used in wearable devices for non-invasive heart rate variability (HRV) monitoring. While most prior work focuses on mitigating motion artifacts, recent studies highlight that even subtle contact pressure variations can distort waveform morphology and lead to inaccurate HRV estimates. In this work, we propose a morphology-aware deep learning framework that conditions HRV estimation on beat-level waveform types. Our model jointly encodes the raw PPG waveform and a sequence of pressure-induced morphology labels using parallel encoders, integrates them via cross-attention, and predicts normal-to-normal (NN) intervals and beat count to support downstream HRV computation. Evaluated on the public …

Developing A Strong Cps Defender: An Evolutionary Approach, Qingyuan Hu, Christopher M. Poskitt, Jun Sun, Yuqi Chen

Research Collection School Of Computing and Information Systems

Cyber-physical systems (CPSs) are used extensively in critical infrastructure, underscoring the need for anomaly detection systems that are able to catch even the most motivated attackers. Traditional anomaly detection techniques typically do `one-off' training on datasets crafted by experts or generated by fuzzers, potentially limiting their ability to generalize to unseen and more subtle attack strategies. Stopping at this point misses a key opportunity: a defender can actively challenge the attacker to find more nuanced attacks, which in turn can lead to more effective detection capabilities. Building on this concept, we propose Evo-Defender, an evolutionary framework that iteratively strengthens CPS …

Better Digital Contracts With Prosocial Friction-In-Design, Brett Frischmann, Moshe Y. Vardi

Faculty Publications

Contract law is supposed to enable people to reach genuine agreements and cooperate. If this ideal was ever a reality, the rise of mass market contracts and boil­erplate rendered it pure fiction. Modern consumer contracts are incomprehensible to most people. No one reads them anyway.

Digital contracting involves design features that amplify traditional boilerplate harms and create others. For example, digital contracting is too cheap; low marginal costs lead to overexpansion in scale and scope. To make matters worse, the loss of autonomy from repeat engagement with digital contracting systems is pernicious. People become increasingly predictable and programmable as digital …