Software Engineering Commons^™

Ai Failures In The Eyes Of The Downstream Developer: A First Look At Concerns, Practices, And Challenges, Haoyu Gao, Mansooreh Zahedi, Wenxin Jiang, Hong Yi Lin, James C. Davis, Christoph Treude

Research Collection School Of Computing and Information Systems

With the advancement of AI models, more software systems are adopting AI as a component to facilitate automation. Pre-trained models (PTMs) have become a cornerstone of AI-based software, allowing for rapid integration and development with lower training cost. However, their adoption also introduces failure modes such as data leakage and biased outputs, that may require careful handling by downstream developers. While previous research has proposed taxonomies of these technical concerns and various mitigation strategies, how downstream developers address these issues during the development of general AI-based software when reusing PTMs remains unexplored. Understanding downstream developers’ perspectives is essential because they …

Analyzing Developer Discussions On Eu And Us Privacy Legislation Compliance In Github Repositories, Georgia M. Kapitsaki, Maria Papoutsoglou, Christoph Treude, Ioanna Theophilou

Research Collection School Of Computing and Information Systems

Context: Privacy legislation has impacted the way software systems are developed, prompting practitioners to update their implementations. Specifically, the EU General Data Protection Regulation (GDPR) and the California Consumer Privacy Act (CCPA) have forced the community to focus on users’ data privacy. Objectives: Relying on the vast amount of data on developer issues available in GitHub repositories, our aim is to gather empirical evidence on the issues developers of Open Source Software discuss to comply with privacy legislation. Method: We examined such discussions by mining and analyzing 32,820 issues from GitHub repositories. We partially analyzed the dataset automatically to identify …

Mutation-Based Multi-Agent Test Case Update, Dawei Tian, Jiakun Liu, Yun Peng, Yichen Zhang, Jianlei Chi, Jun Sun, Xiaohong Su

Research Collection School Of Computing and Information Systems

Modern software systems evolve rapidly under CI/CD practices, where tests are critical for quality. However, substantial code changes often render existing test cases obsolete, causing pipeline disruptions, reduced productivity, and compromised quality. Recent automatic test update approaches leverage LLMs to refine test cases via execution feedback and exact-matching context retrieval, prioritizing executability and line coverage but suffering three limitations: (1) neglecting test assertion adequacy, weakening fault detection; (2) relying on coarse line coverage instead of specific uncovered lines/branches; (3) using exact-matching retrieval, which fails for LLM hallucinated queries. To address these, we propose MuMuTestUp, a mutation-guided multi-agent framework with three …

Ddor: Delta Debugging For Explainable Overrefusal Testing And Repair, Qinyan Zhou, Peixin Zhang, Jun Sun, Haonan Zhang, Dongxia Wang

Research Collection School Of Computing and Information Systems

While safety alignment and guardrails help large language models (LLMs) avoid harmful outputs, they can also induce overrefusal, i.e., unwarranted rejection of benign queries that merely appear risky. We present DDOR (Delta Debugging for OverRefusal), a fully automated and explainable framework for overrefusal testing and repair in a black-box setting, where only model inputs and outputs are accessible and internal safety mechanisms remain opaque. DDOR applies delta debugging to localize minimal refusal-triggering fragments (mRTFs) that provide phrase-level, explainable evidence for why a refusal occurs. Conditioned on these mRTFs, DDOR generates diverse, context-rich prompts and performs multi-oracle validation to filter intrinsically …

Ecu-Malnett V2, Matthew G. Gaber, Mohiuddin Ahmed, Michael N. Johnstone

Research Datasets

ECU-MALNETT (ECU MALware NETwork Traffic) is a real world, reproducible dataset of labeled benign and malicious network flows built from the Peekaboo execution corpus. Peekaboo runs evasive malware with dynamic binary instrumentation and records raw host-level PCAPs while granting full Internet access, yielding noisy, real-world captures with background OS activity and concurrent processes. To derive trustworthy labels from these traces, we apply Construct, a baseline aware, zero-trust labeling framework. Construct first ingests a baseline capture to establish reference sets (DNS qnames, HTTP hosts, TLS SNIs, and socket endpoints) and grows a conservative benign IP pool only via whitelisted DNS resolutions. …

Bayesian And Multi-Objective Decision Support For Incident Mitigation In Cyber-Physical Systems, Shaofei Huang, Christopher M. Poskitt, Lwin Khin Shar

Research Collection School of Computing and Information Systems

Cyber-physical systems increasingly rely on interconnected physical and digital systems whose security incidents can escalate rapidly into safety and operational failures. Existing decision-support approaches struggle to support incident response because they rely on static assumptions, incomplete vulnerability data, and single-objective risk models that do not adequately capture trade-offs between attack success likelihood, impact severity, and system availability. This paper proposes an adaptive decision-support framework for incident mitigation in cyber-physical systems that integrates hierarchical Bayesian Network modelling, confidence-calibrated exposure estimation, and multi-objective optimisation into a unified, adaptive pipeline. The framework constructs probabilistic models from system architecture and vulnerability data, incorporating complementary …

Llm-As-A-Judge For Software Engineering: Literature Review, Vision, And The Road Ahead, Junda He, Jieke Shi, Terry Yue Zhuo, Christoph Treude, Jiamou Sun, Zhenchang Xing, Xiaoning Du, David Lo

Research Collection School Of Computing and Information Systems

The rapid integration of Large Language Models (LLMs) into software engineering (SE) has revolutionized tasks from code generation to program repair, producing a massive volume of software artifacts. This surge in automated creation has exposed a critical bottleneck: the lack of scalable and reliable methods to evaluate the quality of these outputs. Human evaluation, while effective, is very costly and time-consuming. Traditional automated metrics like BLEU rely on high-quality references and struggle to capture nuanced aspects of software quality, such as readability and usefulness. In response, the LLM-as-a-Judge paradigm, which employs LLMs for automated evaluation, has emerged. This approach leverages …

Configuring Agentic Ai Coding Tools: An Exploratory Study, Matthias Galster, Seyedmoein Mohsenimofidi, Jai Lal Lulla, Muhammad Auwal Abubakar, Christoph Treude, Sebastian Baltes

Research Collection School Of Computing and Information Systems

Agentic AI coding tools increasingly automate software development tasks. Developers can configure these tools through versioned repository-level artifacts such as Markdown and JSON files. We present a systematic analysis of configuration mechanisms for agentic AI coding tools, covering Claude Code, GitHub Copilot, Cursor, Gemini, and Codex. We identify eight configuration mechanisms spanning from static context to executable and external integrations and, in an empirical study of 2,853 GitHub repositories, examine whether and how they are adopted, with a detailed analysis of Context Files, Skills, and Subagents. First, Context Files dominate the configuration landscape and are often the sole mechanism in …

Anomaly Management In Unmanned Aerial Vehicles: A Systematic Literature Review, Ivan Tan Wei Han, Christopher M. Poskitt, Lingxiao Jiang, Lwin Khin Shar

Research Collection School Of Computing and Information Systems

Unmanned Aerial Vehicles (UAVs) are increasingly deployed in safety-critical applications such as logistics, surveillance, disaster response, and urban air mobility. While their autonomy enables powerful capabilities, it also introduces vulnerabilities due to hardware faults, software defects, communication failures, and adversarial interference. This survey presents a comprehensive review of research studies closely related to UAV anomalies published between 2015 and 2025, covering 111 papers from academic and industrial sources. We introduce a unified five-pillar taxonomy—anomaly generation, prevention, detection, recovery, and analysis—that organizes existing work across the full anomaly management lifecycle. In contrast to prior surveys that focus primarily on detection algorithms, …

Survey On Learning-Based Dynamic Fault Localization: From Traditional Machine Learning To Large Language Models, Chunyan Liu, Yan Lei, Huan Xie, Jinping Wang, Yue Yu, David Lo

Research Collection School Of Computing and Information Systems

Learning-based dynamic fault localization techniques play a crucial role in the field of software engineering. These techniques dynamically execute test cases to meticulously extract useful knowledge from the execution information in the program, with the aim of identifying fault locations by leveraging machine learning, deep learning, and large language models. Currently, there is already a flourishing body of research that is intensely focused on learning-based dynamic fault localization. Research literature can be categorized into two main aspects for learning-based dynamic fault localization: data-based enhancements (i.e., the datasets) and model-based enhancements (i.e., the suspiciousness algorithms). Thus, we conduct an extensive literature …

Accountable Agents In Software Engineering: An Analysis Of Terms Of Service And A Research Roadmap, Christoph Treude

Research Collection School Of Computing and Information Systems

AI coding assistants and autonomous agents are becoming integral to software development workflows, reshaping how code is produced, reviewed, and maintained. While recent research has focused mainly on the capabilities and impacts of productivity of these systems, much less attention has been paid to accountability: who is responsible when agents generate, modify, or recommend code? In practice, accountability is defined through the Terms of Service (ToS) and related policy documents that govern the use of AI-powered development tools.In this vision paper, we present a comparative analysis of the Terms of Service for widely used AI coding assistants and agent-enabled development …

Operationalizing Ethics For Ai Agents: How Developers Encode Values Into Repository Context Files, Christoph Treude, Sebastian Baltes, Marc Cheong

Research Collection School Of Computing and Information Systems

As AI coding agents become embedded in software development workflows, developers are beginning to operationalize ethical principles by encoding behavioral rules into repository-level context files for AI agents, such as AGENTS.md files. Rather than examining the ethics of AI agents in the abstract, this vision paper investigates how ethics and values are already being translated for AI agents into actionable instructions that shape agent behavior. Through a preliminary investigation, we find that developers are already embedding guidance related to fairness, accessibility, sustainability, tone, and privacy. These artifacts function as a developer-authored governance layer, translating abstract principles into situated, natural-language directives …

A Dataset Of Agentic Ai Coding Tool Configurations, Matthias Galster, Seyedmoein Mohsenimofidi, Levi Böhme, Jai Lal Lulla, Muhammad Auwal Abubakar, Christoph Treude, Sebastian Baltes

Research Collection School Of Computing and Information Systems

Agentic AI coding tools such as Claude Code and OpenAI Codex execute multi-step coding tasks with limited human oversight. To steer these tools, developers create repository-level configuration artifacts (e.g., Markdown files) for configuration mechanisms such as Context Files, Skills, Rules, and Hooks. There is no curated dataset yet that captures these configurations at scale. This dataset, collected from open-source GitHub repositories, fills that gap. We selected 40,585 actively maintained repositories through metadata filtering, classified them using GPT-5.2 to identify 36,710 as belonging to engineered software projects, and systematically detected configuration artifacts in these repositories. The dataset covers 4,738 repositories across …

Train In Vain: Functionality-Preserving Poisoning To Prevent Unauthorized Use Of Code Datasets, Yuan Xiao, Yuchen Chen, Jiaming Wang, Wei Song, Jun Sun, Shiqing Ma, Yanzhou Mu, Juan Zhai, Chunrong Fang, Jin Song Dong, Zhenyu Chen

Research Collection School Of Computing and Information Systems

The widespread availability of large-scale code datasets has accelerated the development of code large language models (CodeLLMs), raising concerns about unauthorized dataset usage. Dataset poisoning offers a proactive defense by reducing the utility of such unauthorized training. However, existing poisoning methods often require full-dataset poisoning and introduce transformations that break code compilability. In this paper, we introduce FunPoison, a functionality-preserving poisoning approach that injects short, compilable weak-use fragments into executed code paths. FunPoison leverages reusable statement-level templates with automatic repair and conservative safety checking to ensure side-effect freedom, while a type-aware synthesis module preserves type correctness, suppresses static-analysis warnings, and …

Activity Transition Graph Generation: How Far Are We?, Jiakun Liu, Peixin Zhang, Han Hu, Yonghui Liu, Wei Minn, Ferdian Thung, Shahar Maoz, Eran Toch, Debin Gao, David Lo

Research Collection School Of Computing and Information Systems

Android applications (i.e., apps) are indispensable nowadays and are getting bigger and bigger with an increasing number offunctionalities. To understand how to access functionalities in an app, prior studies proposed tools to model the transitionsbetween functionalities with the activity transition graph (ATG). ATG is an important data structure and has been used forvarious Android app analyses, including app design, understanding, and testing. However, there is no benchmarking work onATG generation. It is still unclear whether the transitions identified by tools are correct and how many transitions are missed.To fill this gap, we manually identified all transitions in 98 applications to …

How Do Machine Learning Models Change?, Joel Castaño, Rafael Cabañas, Antonio Salmerón, David Lo, Silverio Martínez-Fernández

Research Collection School Of Computing and Information Systems

The proliferation of Machine Learning (ML) models and their open source implementations has transformed AI research and applications. Platforms like Hugging Face (HF) enable this evolving ecosystem, yet a large-scale longitudinal study of how these models change is lacking. This study addresses this gap by analyzing over 680,000 commits from 100,000 models and 2,251 releases from 202 of these models on HF using repository mining and longitudinal methods. We apply an extended ML change taxonomy to classify commits and use Bayesian networks to model temporal patterns in commit and release activities. Our findings show that commit activities align with established …

Patchfuzz: Patch Fuzzing For Javascript Engines, Junjie Wang, Zhihua Xie, Xiaofei Xie, Xiaoning Du, Xiangwei Zhang

Research Collection School Of Computing and Information Systems

Context: Patch fuzzing is a technique aimed at identifying vulnerabilities that arise from newly patched code. While researchers have made efforts to apply patch fuzzing to testing JavaScript (JS) engines with considerable success, these efforts have been limited to using ordinary test cases or publicly available vulnerability PoCs (Proof of Concepts) as seeds, and the sustainability of these approaches is hindered by the challenges associated with automating the PoC collection. Objective: To address these limitations, we propose an end-to-end sustainable approach for JS engine patch fuzzing, named PatchFuzz. Method: It automates the collection of PoCs of a broader range of …

Hydpn: A Hybrid Deep Reinforcement Learning, Programming, And Neighborhood Operations Framework For Integrated Scheduling On Parallel Batch Processing Machines, Yuqi Wang, He Luo, Guoqiang Wang, Zhaoxia Wang

Research Collection School Of Computing and Information Systems

Batch processing machines (BPMs) are widely used in industries such as semiconductors, metal processing, and healthcare, where jobs are processed in batches. As production, inventory, and distribution become increasingly integrated to improve efficiency, research on their joint scheduling in parallel BPM environments remains scarce. This paper addresses the integrated scheduling problem in parallel BPMs, involving production, inventory, and distribution stages, with the objective of minimizing total costs. A unified cost-based model is first formulated, applicable to both in-facility and external distribution scenarios. A hybrid algorithm framework, HyDPN, combining deep reinforcement learning, dynamic programming, and neighborhood operations is proposed. Extensive experiments …

On-The-Fly Generation-Quality Enhancement Of Deep Code Models Via Model Collaboration, Weifeng Sun, Naiqi Huang, Meng Yan, Zhongxin Liu, Hongyan Li, Yan Lei, David Lo

Research Collection School Of Computing and Information Systems

The growing prominence of deep code models in automating software engineering tasks is undeniable. However, their deployment encounters significant challenges in on-the-fly performance enhancement, which refers to dynamically improving the performance of deep code models during real-time execution. Conventional techniques, such as retraining or fine-tuning, are effective in controlled pre-deployment scenarios but fall short when adapting to on-the-fly adjustments post-deployment. CodeDenoise, a notable on-the-fly performance enhancement technology, leverages uncertainty-based methods to identify misclassified inputs and applies an input modification strategy to rectify classification errors. While effective for classification tasks, this approach is inapplicable to generative tasks due to two key …

Hide-And-Sweep: Detecting Concealed Cameras Via Led Illumination Sweeps, Jonghyuk Yun, Jaeyoung Moon, Yunseo Park, Sean Rui Xiang Tan, Byunghyun Kim, Rajesh Krishna Balan, Jun Han

Research Collection School Of Computing and Information Systems

Hidden cameras have increasingly infiltrated hotel and Airbnb rooms, posing serious privacy risks. Detecting such cameras is challenging because they are visually inconspicuous and often embedded inside everyday objects. Even worse, existing handheld detectors are manual and also rely on single-angle illumination and hence suffer from high false-positive rates. We present SweepLED (pronounced "sweepled")1, a practical hidden camera detection system that operates on a commodity smartphone augmented with an unobtrusive LED-embedded case. SweepLED performs LED sweeping - a controlled sequence of multi-angle illumination - while the user simply holds the phone still by hand, enabling the camera to capture how …

“Alexa, Do Not Say That In Front Of My Boss!” A Cross-Cultural Comparison Of User And Ai Preferences For Privacy-Aware Smart Speaker Interactions Across Contexts, Lynne Warin, Emily Aurelia, Anthony Tang, Emily Aurelia, Delphine Reinhardt

Research Collection School Of Computing and Information Systems

Due to their limited ability to reason about the social context in which they are used, smart speakers pose significant privacy risks by responding in ways that may violate people's implicit social boundaries. We conducted a cross-cultural vignette study (N = 944) in Germany and Singapore to investigate how situational factors—specifically social context (bystander relationships and closeness), physical context (location), and interaction context (topic and deceptive intent)—regulate user preferences for smart speaker responses. Our results demonstrate that these factors are superior predictors of response preferences than dispositional user traits (i.e., intrinsic personal traits). We identify two distinct social dynamics: a …

Development And Launch Of Abn: An Adventist Freelance Web Application, Abishur Moses-Pakkianathan

MS in Computer Science Project Reports

The Seventh-day Adventist community often relies on personal networks and word-of-mouth for Adventist business services. Many of these businesses meet and provide services to their clients primarily through church connections and community recommendations. As the community and businesses grow, traditional networking methods become increasingly difficult to maintain. General marketplaces lack the trust framework shared by Adventists, and faith-based directories are often static or outdated without booking capabilities. Our solution provides a reliable and modern platform for Adventists to create service listings, receive and manage bookings, and connect with faith-aligned clients.

Developing Narrative-Based Stem Learning Tool For K-6 Visually Impaired Students, Daniel Tsivkovski, Dylan Ravel, Jeffrey Kraskouskas, Brandon Foley, Maryam Etezad, Franceli Cibrian, Rajeev Joshi, Ariel Han

Student Scholar Symposium Abstracts and Posters

This research develops a free, accessible web application that enables K-6 students who are blind or visually impaired (BVI) to learn STEM concepts using refreshable braille displays. Currently, most online learning tools are not designed for BVI students, creating a significant educational barrier.

The application interfaces with commercial braille displays and uses narrative-based learning to make STEM content approachable and engaging. By presenting material as personalized interactive stories generated with the help of Artificial Intellligence (AI), students can connect with concepts while developing braille reading skills. The curriculum design prioritizes accessibility through the Accessible Rich Internet Applications (ARIA) standards and …

Online Legislation: Developments And Trends In Data Privacy And Software Development, Garrett J. Splinter

Honors Program: Senior Projects (Public)

The increasingly relevant interaction between the law and data privacy, as well as compliance requirements enforced by the United States, is currently in a state of transition. Various states, such as California, Colorado, Connecticut, Nebraska, and many others, all have passed legislation with varied requirements and definitions that make for a challenging framework in which software developers operate, as the universal nature of the internet renders their compliance with current legislation a challenge. Enforcement also tends to be relatively relaxed in most modern examples, though it has escalated after 2020, and this trend may continue in the future, lending credence …

Software Integration In Personal Healthcare Devices And The Patient User Experience, Yassine Chahid, Patrick Slattery

Publications and Research

This study examines the current landscape and future direction of medical device hardware and software integration, focusing on how each contributes to patient care. It begins by analyzing hardware focused medical devices, such as implantable tools patients may rely on to assist with their condition, alongside diagnostic and monitoring equipment used to treat conditions in a variety of medical areas (e.g. cardiovascular conditions). It then evaluates how software is currently integrated through embedded systems, data processing, and user interfaces that support real time monitoring and clinical decision making, and how this impacts quality of care for the patient whilst minimizing …

Func: Reducing The Impact Of Android Framework Evolution On Malware Detection, Hailong Yu, Tiantian Wang, Lwin Khin Shar, Hanmeng Li, David Lo

Research Collection School Of Computing and Information Systems

Android malware detection approaches commonly use APIs and permissions as features for classifying malware. However, since the release of the first Android operating system in 2008, the Android framework has undergone numerous version updates. The evolution of the Android framework over time has led to changes in APIs and permissions, including deprecations and replacements. These changes can result in inaccurate characterization of Android malware, thereby affecting performance of malware detectors. There is a lack of methods to mitigate the impact of Android framework evolution on malware detection. To fill this gap, we conduct a systematic study of the impact of …

Know Thy Enemy: Building A Command-And-Control Solution For Adversarial Emulation, Caleb J. Chen

Senior Honors Theses

Command and Control (C2) is a critical part of any cyberattack. It serves many purposes, including Distributed Denial of Service (DDoS) attacks, data exfiltration, and malware deployment. Consequently, C2 frameworks play an important part in red team engagements and adversary emulation. However, many adversary emulation solutions focus on comprehensive testing through sequential technique execution instead of realistic chained and automated attacks. The proposed solution is Centurion, an open-source C2 framework that integrates MITRE's ATT&CK framework and several cybersecurity tools into modular playbooks for effective threat emulation. This paper provides background by defining key terms and concepts before delving into a …

Open Source Software Development Tool Installation: Challenges And Strategies For Novice Developers, Larissa Salerno, Christoph Treude, Patanamon Thongtanunam

Research Collection School Of Computing and Information Systems

As the world of technology advances, so do the tools that software developers use to create new programs. In recent years, software development tools have become more popular, allowing developers to work more efficiently and produce higher-quality software. Still, installing such tools can be challenging for novice developers at the early stage of their careers, as they may face issues such as compatibility problems (e.g., with operating systems) and unclear instructions. Therefore, this work aims to investigate the challenges novice developers face when installing software development tools and the strategies they employ to overcome them. To investigate these, we conducted …

Natural Adversaries: Fuzzing Autonomous Vehicles With Realistic Roadside Object Placements, Yang Sun, Haoyu Wang, Christopher M. Poskitt, Jun Sun

Research Collection School Of Computing and Information Systems

The emergence of Autonomous Vehicles (AVs) has spurred research into testing the resilience of their perception systems, i.e., ensuring that they are not susceptible to critical misjudgements. It is important that these systems are tested not only with respect to other vehicles on the road, but also with respect to objects placed on the roadside. Trash bins, billboards, and greenery are examples of such objects, typically positioned according to guidelines developed for the human visual system, which may not align perfectly with the needs of AVs. Existing tests, however, usually focus on adversarial objects with conspicuous shapes or patches, which …

Selective Concolic Testing, Guofeng Zhang, Zhenbang Chen, Ziqi Shuai, Jun Sun, Weijiang Hong, Yufeng Zhang, Ji Wang, Yang Liu

Research Collection School Of Computing and Information Systems

The principled combination of symbolic execution and random testing lacks a formal foundation, especially in deciding which inputs to symbolize. We propose selective concolic testing, a cost-aware framework that formulates this choice as an optimized policy problem of a MDP (Markov Decision Process). We model program exploration over a finite control-flow graph, where MDP states represent covered statements, actions partition path constraints into symbolic and random fragments, rewards reflect coverage gain, and costs account for SMT solving effort and sampling inefficiency. Our framework yields the first formal characterization of selective symbolization as policy synthesis in a probabilistic system. We prove …