Software Engineering Commons^™

Llm-As-A-Judge For Software Engineering: Literature Review, Vision, And The Road Ahead, Junda He, Jieke Shi, Terry Yue Zhuo, Christoph Treude, Jiamou Sun, Zhenchang Xing, Xiaoning Du, David Lo

Research Collection School Of Computing and Information Systems

The rapid integration of Large Language Models (LLMs) into software engineering (SE) has revolutionized tasks from code generation to program repair, producing a massive volume of software artifacts. This surge in automated creation has exposed a critical bottleneck: the lack of scalable and reliable methods to evaluate the quality of these outputs. Human evaluation, while effective, is very costly and time-consuming. Traditional automated metrics like BLEU rely on high-quality references and struggle to capture nuanced aspects of software quality, such as readability and usefulness. In response, the LLM-as-a-Judge paradigm, which employs LLMs for automated evaluation, has emerged. This approach leverages …

Pharmed. System Manual: Hololens Application, Eden Woldeselassie

Systems Manuals - 2026

This document provides the full life cycle of PharmEd HoloLens. PharmEd stands for Pharmacy Education. The overall document will go through the complete system manual by going over requirements, followed by design, how to install the application, how to upload the application to Microsoft store, troubleshooting and source code. The first section will explain the background, the existing problems and why the need to develop PharmEd.  Here techniques of requirement gathering and some statistics will elaborated. The following section discusses about functional requirements. The requirement section includes subtopics: functional and nonfunctional requirements for both -programmer and end user- and the …

Development And Launch Of Abn: An Adventist Freelance Web Application, Abishur Moses-Pakkianathan

MS in Computer Science Project Reports

The Seventh-day Adventist community often relies on personal networks and word-of-mouth for Adventist business services. Many of these businesses meet and provide services to their clients primarily through church connections and community recommendations. As the community and businesses grow, traditional networking methods become increasingly difficult to maintain. General marketplaces lack the trust framework shared by Adventists, and faith-based directories are often static or outdated without booking capabilities. Our solution provides a reliable and modern platform for Adventists to create service listings, receive and manage bookings, and connect with faith-aligned clients.

Pyspqr: A Python Package For Density Estimation Using Deep Learning, Cameron Eddy, Reetam Majumder

Electrical Engineering and Computer Science Undergraduate Honors Theses

Splines are used for representing complex functions. In statistics, splines can be used for distributional shapes that are difficult to model by traditional parametric approaches. Ramsay (1) uses M-Spline bases to estimate continuous distributions. Semi-Parametric Quantile Regression (SPQR), developed by Xu and Reich (2), models conditional distributions where a neural network is used to estimate the basis function weights that depend on covariates. (3) implements a package for SPQR in R. We build on this by implementing a version of SPQR in Python with PyTorch. By using PyTorch, we can use more sophisticated deep learning architectures than those available in …

Context Engineering For Ai Agents In Open-Source Software, Seyedmoein Mohsenimofidi, Matthias Galster, Christoph Treude, Sebastian Baltes

Research Collection School Of Computing and Information Systems

GenAI-based coding assistants have disrupted software development. The next generation of these tools is agent-based, operating with more autonomy and potentially without human oversight. Like human developers, AI agents require contextual information to develop solutions that are in line with the standards, policies, and workflows of the software projects they operate in. Vendors of popular agentic tools (e.g., Claude Code) recommend maintaining version-controlled Markdown files that describe aspects such as the project structure, code style, or building and testing. The content of these files is then automatically added to each prompt. Recently, AGENTS.md has emerged as a potential standard that …

Software Engineering In The Age Of Coding Agents: Failure Modes And Rejection Patterns, Mahd Mohd Taysir Hindi

Thesis/ Dissertation Defenses

This thesis investigates the real-world behavior of LLM-driven coding agents that generate code changes and submit pull requests (PRs) to public software repositories. As these tools evolve from autocomplete-style assistants into more autonomous agents, their contributions increasingly interact with socio-technical review processes (human reviewers, bots, CI/CD gates, and project norms). This thesis focuses on understanding why agent-generated PRs are accepted or rejected and what these outcomes reveal about current agent limitations in practical development workflows. The main objective of this thesis is to systematically characterize rejection patterns and failure modes of agent-generated pull requests in real repositories. Specifically, the thesis …

Simulated Interactive Debugging, Yannic Noller, Erick Chandra, Srinidhi Chandrashekar, Kenny Choo, Cyrille Jegourel, Oka Kurniawan, Christopher M. Poskitt

Research Collection School Of Computing and Information Systems

Debugging software, i.e., the localization of faults and their repair, is a key activity in software engineering. Therefore, effective and efficient debugging is one of the core skills a software engineer must develop. However, the teaching of debugging techniques is usually very limited or only taught in indirect ways, e.g., during software projects. As a result, most Computer Science (CS) students learn debugging only in an ad-hoc and unstructured way. In this work, we present our approach called Simulated Interactive Debugging that interactively guides students along the debugging process. The guidance aims to empower the students to repair their solutions …

Generative Ai And Empirical Software Engineering: A Paradigm Shift, Christoph Treude, Margaret-Anne Storey

Research Collection School Of Computing and Information Systems

The widespread adoption of generative AI in software engineering marks a paradigm shift, offering new opportunities to design and utilize software engineering tools while influencing both developers and the artifacts they create. Traditional empirical methods in software engineering, including quantitative, qualitative, and mixed-method approaches, are well established. However, this paradigm shift introduces novel data types and redefines many concepts in the software engineering process. The roles of developers, users, agents, and researchers increasingly overlap, blurring the distinctions between these social and technical actors within the field. This paper examines how integrating AI into software engineering challenges traditional research paradigms. It …

Llm-Based Multi-Agent Systems For Software Engineering: Literature Review, Vision And The Road Ahead, Junda He, Christoph Treude, David Lo

Research Collection School Of Computing and Information Systems

Integrating Large Language Models (LLMs) into autonomous agents marks a significant shift in the research landscape by offering cognitive abilities that are competitive with human planning and reasoning. This paper explores the transformative potential of integrating Large Language Models into Multi-Agent (LMA) systems for addressing complex challenges in software engineering (SE). By leveraging the collaborative and specialized abilities of multiple agents, LMA systems enable autonomous problem-solving, improve robustness, and provide scalable solutions for managing the complexity of real-world software projects. In this paper, we conduct a systematic review of recent primary studies to map the current landscape of LMA applications …

Efficient And Green Large Language Models For Software Engineering: Literature Review, Vision, And The Road Ahead, Jieke Shi, Zhou Yang, David Lo

Research Collection School Of Computing and Information Systems

Large Language Models (LLMs) have recently shown remarkable capabilities in various software engineering tasks, spurring the rapid growth of the Large Language Models for Software Engineering (LLM4SE) area. However, limited attention has been paid to developing efficient LLM4SE techniques that demand minimal computational cost, time, and memory resources, as well as green LLM4SE solutions that reduce energy consumption, water usage, and carbon emissions. This article aims to redirect the focus of the research community toward the efficiency and greenness of LLM4SE, while also sharing potential research directions to achieve this goal. It commences with a brief overview of the significance …

Applying Software Engineering Black-Box Methods For Testing Machine Learning Models, Timothy Elvira

Doctoral Dissertations and Master's Theses

This dissertation proposes researching an approach to incorporate and align Software black-box testing methods into Machine Learning (ML) applications, specifically in the context of computer vision models. Typically, testing methods within Software Engineering (SE) encompass a range of test types that assess levels of a software system, such as Unit, Integration, Functional, and System testing [1]. The testing spectrum offers two perspectives on the system: black-box, where the system’s code is hidden, and white-box, where the system's code is exposed for testing. Software Quality pairs testing with requirements, in a many-to-one relationship, to ensure proper validation of the software system. …

Revisiting Sentiment Analysis For Software Engineering In The Era Of Large Language Models, Ting Zhang, Ivana Clairine Irsan, Thung Ferdian, David Lo

Research Collection School Of Computing and Information Systems

Software development involves collaborative interactions where stakeholders express opinions across various platforms. Recognizing the sentiments conveyed in these interactions is crucial for the effective development and ongoing maintenance of software systems. For software products, analyzing the sentiment of user feedback, e.g., reviews, comments, and forum posts can provide valuable insights into user satisfaction and areas for improvement. This can guide the development of future updates and features. However, accurately identifying sentiments in software engineering datasets remains challenging.This study investigates bigger large language models (bLLMs) in addressing the labeled data shortage that hampers fine-tuned smaller large language models (sLLMs) in software …

Large Language Models For Software Engineering: A Systematic Literature Review, Xinyi Hou, Yanjie Zhao, Yue Liu, Zhou Yang, Kailong Wang, Li Li, Xiapu Luo, David Lo, John Grundy, Haoyu Wang

Research Collection School Of Computing and Information Systems

Large Language Models (LLMs) have significantly impacted numerous domains, including Software Engineering (SE). Many recent publications have explored LLMs applied to various SE tasks. Nevertheless, a comprehensive understanding of the application, effects, and possible limitations of LLMs on SE is still in its early stages. To bridge this gap, we conducted a Systematic Literature Review (SLR) on LLM4SE, with a particular focus on understanding how LLMs can be exploited to optimize processes and outcomes. We selected and analyzed 395 research articles from January 2017 to January 2024 to answer four key Research Questions (RQs). In RQ1, we categorize different LLMs …

Can Large-Language Models Help Us Better Understand And Teach The Development Of Energy-Efficient Software?, Ryan Hasler, Konstantin Laufer, George K. Thiruvathukal, Huiyun Peng, Kyle Robinson, Kirsten Davis, Yung-Hisang Lu, James C. Davis

Computer Science: Faculty Publications and Other Works

Computing systems are consuming an increasing and unsustainable fraction of society's energy footprint, notably in data centers. Meanwhile, energy-efficient software engineering techniques are often absent from undergraduate curricula. We propose to develop a learning module for energy-efficient software, suitable for incorporation into an undergraduate software engineering class. There is one major problem with such an endeavor: undergraduate curricula have limited space for mastering energy-related systems programming aspects. To address this problem, we propose to leverage the domain expertise afforded by large language models (LLMs). In our preliminary studies, we observe that LLMs can generate energy-efficient variations of basic linear algebra …

A Ui-Enhanced Approach To Generic Web-Based Scheduling, Tyler Hinrichs

Honors Scholar Theses

Administrative scheduling is a key aspect of a wide variety of systems, but despite being a widespread need, it is not a straightforward task. Organizational uniqueness introduces complexity when attempting to use algorithmic methods to automate scheduling, as individual organizations often have their own ways of determining various details and constraints of a schedule. However, in this paper, we assert that there are relevant commonalities that many different schedules fundamentally possess, allowing us to create a generic scheduling application that can be productively used for as many different scenarios as possible. After devising a schema that captures this generic representation, …

Elevating Academic Administration: A Comprehensive Faculty Dashboard For Tracking Student Evaluations And Research, Musa M. Azeem

Senior Theses

The USC Faculty Dashboard is a web application designed to revolutionize how department heads, professors, and instructors monitor progress and make decisions, providing a centralized hub for efficient data storage and analysis. Currently, there’s a gap in tools tailored for department heads to concisely manage the performance of their department, which our platform aims to fill. The USC Faculty Dashboard offers easy access to upload and view student evaluation and research information, empowering department heads to evaluate the performance of faculty members and seamlessly track their research grants, publications, and expenditures. Furthermore, professors and instructors gain personalized performance analysis tools, …

Towards Effective Developer Communication In Open Source Software Via Emotional Awareness, Mia Mohammad Imran

Theses and Dissertations

Emotions play an integral yet understudied role in open-source software development, profoundly shaping critical collaborative processes such as knowledge sharing, decision-making, and team dynamics. However, accurately detecting and analyzing emotions in developer communications poses significant challenges due to the lack of visual and auditory cues in text-based interactions. This dissertation investigates techniques to enhance the understanding and modeling of emotions within the textual artifacts of open-source projects. We conduct an extensive evaluation of existing emotion classification tools using a novel dataset of annotated GitHub comments. An error analysis reveals deficiencies in handling implicit emotional expressions and figurative language. We demonstrate …

Learning Representations For Effective And Explainable Software Bug Detection And Fixing, Yi Li

Dissertations

Software has an integral role in modern life; hence software bugs, which undermine software quality and reliability, have substantial societal and economic implications. The advent of machine learning and deep learning in software engineering has led to major advances in bug detection and fixing approaches, yet they fall short of desired precision and recall. This shortfall arises from the absence of a 'bridge,' known as learning code representations, that can transform information from source code into a suitable representation for effective processing via machine and deep learning.

This dissertation builds such a bridge. Specifically, it presents solutions for effectively learning …

Designing Programming Languages For Writing Maintainable Software, Aaron Friesen

Honors Program: Senior Projects (Public)

Maintainability is crucial to the long-term success of software projects. Among other factors, it is affected by the programming language in which the software is written. Programming language designers should be conscious of how their design decisions can influence software maintainability. Non-functional properties of a language can affect the readability of source code in ways beyond the control of programmers. Language features can cause or prevent certain classes of bugs, and runtime issues especially can require significant maintenance effort. Tools external to the language, especially those developed and distributed by language implementers, can aid in the creation of maintainable software. …

Fortifying The Seams Of Software Systems, Hong Jin Kang

Dissertations and Theses Collection (Open Access)

A seam in software is a place where two components within a software system meet. There are more seams in software now than ever before as modern software systems rely extensively on third-party software components, e.g., libraries. Due to the increasing complexity of software systems, understanding and improving the reliability of these components and their use is crucial. While the use of software components eases the development process, it also introduces challenges due to the interaction between the components.

This dissertation tackles problems associated with software reliability when using third-party software components. Developers write programs that interact with libraries through …

How Developers Engineer Test Cases: An Observational Study, Maurício Aniche, Christoph Treude, Andy Zaidman

Research Collection School Of Computing and Information Systems

One of the main challenges that developers face when testing their systems lies in engineering test cases that are good enough to reveal bugs. And while our body of knowledge on software testing and automated test case generation is already quite significant, in practice, developers are still the ones responsible for engineering test cases manually. Therefore, understanding the developers’ thought- and decision-making processes while engineering test cases is a fundamental step in making developers better at testing software. In this paper, we observe 13 developers thinking-aloud while testing different real-world open-source methods, and use these observations to explain how developers …

Unoapi: Balancing Performance, Portability, And Productivity (P3) In Hpc Education, Konstantin Laufer, George K. Thiruvathukal

Computer Science: Faculty Publications and Other Works

oneAPI is a major initiative by Intel aimed at making it easier to program heterogeneous architectures used in high-performance computing using a unified application programming interface (API). While raising the abstraction level via a unified API represents a promising step for the current generation of students and practitioners to embrace high- performance computing, we argue that a curriculum of well- developed software engineering methods and well-crafted exem- plars will be necessary to ensure interest by this audience and those who teach them. We aim to bridge the gap by developing a curriculum—codenamed UnoAPI—that takes a more holistic approach by looking …

Rasm: Compiling Racket To Webassembly, Grant Matejka

Master's Theses

WebAssembly is an instruction set designed for a stack based virtual machine, with an emphasis on speed, portability and security. As the use cases for WebAssembly grow, so does the desire to target WebAssembly in compilation. In this thesis we present Rasm, a Racket to WebAssembly compiler that compiles a select subset of the top forms of the Racket programming language to WebAssembly. We also present our early findings in our work towards adding a WebAssembly backend to the Chez Scheme compiler that is the backend of Racket. We address initial concerns and roadblocks in adopting a WebAssembly backend and …

Challenges For Inclusion In Software Engineering: The Case Of The Emerging Papua New Guinean Society, Raula Kula, Christoph Treude, Hideaki Hata, Sebastian Baltes, Igor Steinmacher, Marco Gerosa, Winifred Kula Amini

Research Collection School Of Computing and Information Systems

Software plays a central role in modern societies, with its high economic value and potential for advancing societal change. In this paper, we characterise challenges and opportunities for a country progressing towards entering the global software industry, focusing on Papua New Guinea (PNG). By hosting a Software Engineering workshop, we conducted a qualitative study by recording talks (n=3), employing a questionnaire (n=52), and administering an in-depth focus group session with local actors (n=5). Based on a thematic analysis, we identified challenges as barriers and opportunities for the PNG software engineering community. We also discuss the state of practices and how …

Post2vec: Learning Distributed Representations Of Stack Overflow Posts, Bowen Xu, Thong Hoang, Abhishek Sharma, Chengran Yang, Xin Xia, David Lo

Research Collection School Of Computing and Information Systems

Past studies have proposed solutions that analyze Stack Overflow content to help users find desired information or aid various downstream software engineering tasks. A common step performed by those solutions is to extract suitable representations of posts; typically, in the form of meaningful vectors. These vectors are then used for different tasks, for example, tag recommendation, relatedness prediction, post classification, and API recommendation. Intuitively, the quality of the vector representations of posts determines the effectiveness of the solutions in performing the respective tasks. In this work, to aid existing studies that analyze Stack Overflow posts, we propose a specialized deep …

Reflecting On Recurring Failures In Iot Development, Dharun Anandayuvaraj, James C. Davis

Department of Electrical and Computer Engineering Faculty Publications

As IoT systems are given more responsibility and autonomy, they offer greater benefits, but also carry greater risks. We believe this trend invigorates an old challenge of software engineering: how to develop high-risk software-intensive systems safely and securely under market pressures? As a first step, we conducted a systematic analysis of recent IoT failures to identify engineering challenges. We collected and analyzed 22 news reports and studied the sources, impacts, and repair strategies of failures in IoT systems. We observed failure trends both within and across application domains. We also observed that failure themes have persisted over time. To alleviate …

On The Reproducibility And Replicability Of Deep Learning In Software Engineering, Chao Liu, Cuiyun Gao, Xin Xia, David Lo, John C. Grundy, Xiaohu Yang

Research Collection School Of Computing and Information Systems

Context: Deep learning (DL) techniques have gained significant popularity among software engineering (SE) researchers in recent years. This is because they can often solve many SE challenges without enormous manual feature engineering effort and complex domain knowledge.Objective: Although many DL studies have reported substantial advantages over other state-of-the-art models on effectiveness, they often ignore two factors: (1) reproducibility—whether the reported experimental results can be obtained by other researchers using authors’ artifacts (i.e., source code and datasets) with the same experimental setup; and (2) replicability—whether the reported experimental result can be obtained by other researchers using their re-implemented artifacts with a …

On The Documentation Of Refactoring Types, Eman Abdullah Alomar, Jiaqian Liu, Kenneth Addo, Mohamed Wiem Mkaouer, Christian D. Newman, Ali Ouni, Zhe Yu

Articles

Commit messages are the atomic level of software documentation. They provide a natural language description of the code change and its purpose. Messages are critical for software maintenance and program comprehension. Unlike documenting feature updates and bug fixes, little is known about how developers document their refactoring activities. Specifically, developers can perform multiple refactoring operations, including moving methods, extracting classes, renaming attributes, for various reasons, such as improving software quality, managing technical debt, and removing defects. Yet, there is no systematic study that analyzes the extent to which the documentation of refactoring accurately describes the refactoring operations performed at the …

Adopting Agility In Academia Through Pilot Projects, Omar Ochoa, Massood Towhidnejad

Papers

This innovate practice work in progress paper presents our initial findings about the application of agile process in the day to day operation of an academic department. The benefits of agile methods, which are processes that are lightweight and people-oriented, are well-known. Agile methods provide the ability to respond faster to the changing needs of stakeholders, improved product quality, team over individual goals, and better transparency. Therefore, one would expect adaption of the agile process within academic environment would be welcomed, however, unique issues exist within faculty that impact adoption; faculty have different goals that change through their tenure, and …

Utilizing Software Engineering And Cloud Computing Principles To Develop The Revised Self-Report Assessment Of Functional Visual Performance (R-Srafvp) Application, Kirk Hedlich

Masters Projects

Can principles from software engineering and concepts from cloud computing be applied to and aid in the development of a small project, specifically improving the use of the Revised Self-Report Assessment of Functional Visual Performance (R-SRAFVP)? The target for this project is to create a new application and improve on existing attempts to move the R-SRAFVP assessment from an electronic document format to a web-based application. The new application should provide better ease of use, simplicity in design and understanding for use, and hopefully increased access and adoption by Occupation Therapists who specialize in low vision rehabilitation. The benefits of …