Software Engineering Commons^™

Towards Auto-Evaluation For Large Language Models, Jiahao Ying

Dissertations and Theses Collection (Open Access)

The rapid advancement of large language models (LLMs) has created an urgent need for evaluation methodologies that are timely, scalable, reliable, and informative. Conventional evaluation benchmarks, although essential for measuring model capabilities and guiding model development, are often constructed and maintained through labor-intensive human annotation. As LLMs continue to improve through increases in model scale, training data, and computational resources, static benchmarks may quickly lose discriminative power. Moreover, the growing use of large and diverse training corpora increases the risk of benchmark leakage, which can inflate evaluation results and obscure the true capabilities of models. These challenges call for a …

On-The-Fly Generation-Quality Enhancement Of Deep Code Models Via Model Collaboration, Weifeng Sun, Naiqi Huang, Meng Yan, Zhongxin Liu, Hongyan Li, Yan Lei, David Lo

Research Collection School Of Computing and Information Systems

The growing prominence of deep code models in automating software engineering tasks is undeniable. However, their deployment encounters significant challenges in on-the-fly performance enhancement, which refers to dynamically improving the performance of deep code models during real-time execution. Conventional techniques, such as retraining or fine-tuning, are effective in controlled pre-deployment scenarios but fall short when adapting to on-the-fly adjustments post-deployment. CodeDenoise, a notable on-the-fly performance enhancement technology, leverages uncertainty-based methods to identify misclassified inputs and applies an input modification strategy to rectify classification errors. While effective for classification tasks, this approach is inapplicable to generative tasks due to two key …

Political Inconsistency Detection Across Legislative Speech And Public Communications, Scott M. Pramuk

Master's Theses

Political actors communicate about legislation across multiple contexts, including committee hearings, recorded votes, and public-facing press releases. Differences between these forms of communication can provide useful signals for journalists and researchers seeking to understand how legislators present policy positions to different audiences.

This thesis extends the Digital Democracy Project, a legislative transparency initiative that provides access to California state legislative hearing transcripts, voting records, and related legislative data. Specifically, this work incorporates publicly accessible, legislator-authored news releases into the Digital Democracy Database and develops a pipeline for analyzing legislative communication across multiple sources. The system collects news releases from California …

Patchfuzz: Patch Fuzzing For Javascript Engines, Junjie Wang, Zhihua Xie, Xiaofei Xie, Xiaoning Du, Xiangwei Zhang

Research Collection School Of Computing and Information Systems

Context: Patch fuzzing is a technique aimed at identifying vulnerabilities that arise from newly patched code. While researchers have made efforts to apply patch fuzzing to testing JavaScript (JS) engines with considerable success, these efforts have been limited to using ordinary test cases or publicly available vulnerability PoCs (Proof of Concepts) as seeds, and the sustainability of these approaches is hindered by the challenges associated with automating the PoC collection. Objective: To address these limitations, we propose an end-to-end sustainable approach for JS engine patch fuzzing, named PatchFuzz. Method: It automates the collection of PoCs of a broader range of …

Hydpn: A Hybrid Deep Reinforcement Learning, Programming, And Neighborhood Operations Framework For Integrated Scheduling On Parallel Batch Processing Machines, Yuqi Wang, He Luo, Guoqiang Wang, Zhaoxia Wang

Research Collection School Of Computing and Information Systems

Batch processing machines (BPMs) are widely used in industries such as semiconductors, metal processing, and healthcare, where jobs are processed in batches. As production, inventory, and distribution become increasingly integrated to improve efficiency, research on their joint scheduling in parallel BPM environments remains scarce. This paper addresses the integrated scheduling problem in parallel BPMs, involving production, inventory, and distribution stages, with the objective of minimizing total costs. A unified cost-based model is first formulated, applicable to both in-facility and external distribution scenarios. A hybrid algorithm framework, HyDPN, combining deep reinforcement learning, dynamic programming, and neighborhood operations is proposed. Extensive experiments …

“Alexa, Do Not Say That In Front Of My Boss!” A Cross-Cultural Comparison Of User And Ai Preferences For Privacy-Aware Smart Speaker Interactions Across Contexts, Lynne Warin, Emily Aurelia, Anthony Tang, Emily Aurelia, Delphine Reinhardt

Research Collection School Of Computing and Information Systems

Due to their limited ability to reason about the social context in which they are used, smart speakers pose significant privacy risks by responding in ways that may violate people's implicit social boundaries. We conducted a cross-cultural vignette study (N = 944) in Germany and Singapore to investigate how situational factors—specifically social context (bystander relationships and closeness), physical context (location), and interaction context (topic and deceptive intent)—regulate user preferences for smart speaker responses. Our results demonstrate that these factors are superior predictors of response preferences than dispositional user traits (i.e., intrinsic personal traits). We identify two distinct social dynamics: a …

Ai-Driven Vehicular Federated Learning: Fairness-Aware Adaptive Incentives With Blockchain Verifiability For Smart Transportation, Abir Raza

Thesis/ Dissertation Defenses

Vehicular Federated Learning (VFL) is becoming a crucial enabler for the implementation and optimization of automated transport systems. This technology enables intelligent transportation systems to function by enabling networked vehicles to develop perception and control models through joint training while preserving their original data. Nevertheless, the effective deployment of VFL is dependent on the sustained participation of trustworthy vehicles. However, the dynamic nature of vehicular environments poses critical challenges, including unstable participation, data heterogeneity, and resource constraints. The sustained collaboration of smart vehicles necessitates reliable client selection and equitable incentive schemes with verifiable transparency. The distributed nature of VFL makes …

Synthergy: Social Deduction And Deception In Llm-Powered Agents, Lauren Campbell, Andrew Forney

Honors Thesis

Synthergy is an online social deduction game designed to enable comparative analysis of how large language model-powered agents engage in social deduction and deception under conditions of asymmetric information. Inspired by social deduction games such as Town of Salem, Throne of Lies, and Mafia, the game consists of two factions, Harmony and Discord, to which agents are secretly assigned. Agents must infer others’ affiliations through dialogue, in-game abilities, and voting behavior. To evaluate agent behavior, we conducted 100 simulated games across six agent types: a random baseline agent (RandomSynth), an LLM-based agent (Synth), a chain-of-thought agent (CoT Synth), a Bayesian …

Development And Launch Of Abn: An Adventist Freelance Web Application, Abishur Moses-Pakkianathan

MS in Computer Science Project Reports

The Seventh-day Adventist community often relies on personal networks and word-of-mouth for Adventist business services. Many of these businesses meet and provide services to their clients primarily through church connections and community recommendations. As the community and businesses grow, traditional networking methods become increasingly difficult to maintain. General marketplaces lack the trust framework shared by Adventists, and faith-based directories are often static or outdated without booking capabilities. Our solution provides a reliable and modern platform for Adventists to create service listings, receive and manage bookings, and connect with faith-aligned clients.

Nutrilog: Design And Development Of A Full-Stack Web Application For Holistic Health Tracking, Jaromir J. Saloni

Honors Theses

NutriLog is a web-based fitness and nutrition tracking application designed to help users record, organize, and interpret personal health data in one centralized platform. Many existing applications focus primarily on either nutrition tracking or exercise performance, which can make it difficult for users to understand how food intake and physical activity interact. NutriLog addresses this gap by combining food logging, exercise logging, calorie adjustment, nutrition summaries, and historical tracking into a single dashboard-based interface.

The application was developed using React and TypeScript for the frontend and Supabase for authentication, database storage, and user-specific data management. Nutrition data is supported through …

Developing Narrative-Based Stem Learning Tool For K-6 Visually Impaired Students, Daniel Tsivkovski, Dylan Ravel, Jeffrey Kraskouskas, Brandon Foley, Maryam Etezad, Franceli Cibrian, Rajeev Joshi, Ariel Han

Student Scholar Symposium Abstracts and Posters

This research develops a free, accessible web application that enables K-6 students who are blind or visually impaired (BVI) to learn STEM concepts using refreshable braille displays. Currently, most online learning tools are not designed for BVI students, creating a significant educational barrier.

The application interfaces with commercial braille displays and uses narrative-based learning to make STEM content approachable and engaging. By presenting material as personalized interactive stories generated with the help of Artificial Intellligence (AI), students can connect with concepts while developing braille reading skills. The curriculum design prioritizes accessibility through the Accessible Rich Internet Applications (ARIA) standards and …

Online Legislation: Developments And Trends In Data Privacy And Software Development, Garrett J. Splinter

Honors Program: Senior Projects (Public)

The increasingly relevant interaction between the law and data privacy, as well as compliance requirements enforced by the United States, is currently in a state of transition. Various states, such as California, Colorado, Connecticut, Nebraska, and many others, all have passed legislation with varied requirements and definitions that make for a challenging framework in which software developers operate, as the universal nature of the internet renders their compliance with current legislation a challenge. Enforcement also tends to be relatively relaxed in most modern examples, though it has escalated after 2020, and this trend may continue in the future, lending credence …

Pyspqr: A Python Package For Density Estimation Using Deep Learning, Cameron Eddy, Reetam Majumder

Electrical Engineering and Computer Science Undergraduate Honors Theses

Splines are used for representing complex functions. In statistics, splines can be used for distributional shapes that are difficult to model by traditional parametric approaches. Ramsay (1) uses M-Spline bases to estimate continuous distributions. Semi-Parametric Quantile Regression (SPQR), developed by Xu and Reich (2), models conditional distributions where a neural network is used to estimate the basis function weights that depend on covariates. (3) implements a package for SPQR in R. We build on this by implementing a version of SPQR in Python with PyTorch. By using PyTorch, we can use more sophisticated deep learning architectures than those available in …

Software Integration In Personal Healthcare Devices And The Patient User Experience, Yassine Chahid, Patrick Slattery

Publications and Research

This study examines the current landscape and future direction of medical device hardware and software integration, focusing on how each contributes to patient care. It begins by analyzing hardware focused medical devices, such as implantable tools patients may rely on to assist with their condition, alongside diagnostic and monitoring equipment used to treat conditions in a variety of medical areas (e.g. cardiovascular conditions). It then evaluates how software is currently integrated through embedded systems, data processing, and user interfaces that support real time monitoring and clinical decision making, and how this impacts quality of care for the patient whilst minimizing …

Collaborative Practices And Tool Utilization In Software Development Projects: A Student Perspective, Yi Meng Lau, Muhammad Syahmi Bin Abbas, Lingxiao Jiang

Research Collection School Of Computing and Information Systems

Software development is a collaborative activity that depends on effective teamwork, shared understanding, and coordinated use of development practices and tools. While these aspects are well studied in professional environments, they are less frequently examined within software engineering education. This study investigates how students collaborate in group projects, focusing on collaborative practices, tool usage, and their perceptions of software quality. We conducted a quantitative post-project survey with 143 second-year undergraduate students enrolled in a software development course. The results show that students actively share information and often establish team norms to support coordination and collaboration. However, students face challenges in …

Causality-Driven Test Case Minimisation For Cyber-Physical Systems, Michael Foster, Christopher M. Poskitt, Nicholas R. Latimer, Neil Walkinshaw, Richard Somers, Robert M. Hierons

Research Collection School Of Computing and Information Systems

Cyber-physical systems allow digital control systems to interact with the physical world using sensors and actuators. They are increasingly being used to automate critical infrastructure, where software faults can have dire consequences. Due to the complex nature and unpredictability of these systems, their resilience is often tested using a technique called fuzzing, which generates quasi-random sequences of sensor and actuator manipulations with the goal of forcing a system into unsafe states. However, there is currently no way of determining which manipulations of a test case cause a failure without systematically removing each one and re-running the test, which can be …

Func: Reducing The Impact Of Android Framework Evolution On Malware Detection, Hailong Yu, Tiantian Wang, Lwin Khin Shar, Hanmeng Li, David Lo

Research Collection School Of Computing and Information Systems

Android malware detection approaches commonly use APIs and permissions as features for classifying malware. However, since the release of the first Android operating system in 2008, the Android framework has undergone numerous version updates. The evolution of the Android framework over time has led to changes in APIs and permissions, including deprecations and replacements. These changes can result in inaccurate characterization of Android malware, thereby affecting performance of malware detectors. There is a lack of methods to mitigate the impact of Android framework evolution on malware detection. To fill this gap, we conduct a systematic study of the impact of …

Weavecc: Symbolically-Guided Joint Exploration Of Inputs And Schedules For Concurrency Bug Detection, William Philip Dinauer

Dartmouth College Master’s Theses

Concurrent programs introduce a class of bugs that depend jointly on both program inputs and thread schedules. Exposing these bugs requires simultaneously reasoning about which code paths are reachable and which thread interleavings are possible. At the same time, many existing tools handle the problem insufficiently. Race detectors observe only the interleavings that the OS happens to produce. Fuzzers explore inputs without controlling schedules. Tools that address both dimensions together exist, but are built on interpretation-based symbolic executors that incur considerable overhead.

We present WeaveCC, a practical concurrency testing tool for C/C++ programs that jointly explores inputs and thread schedules. …

Sql Query Optimization - Human Vs. Chatgpt, Hailey Dennis

All Graduate Reports and Creative Projects, Fall 2023 to Present

Large Language Models (LLMs) such as ChatGPT have become ubiquitous tools for working professionals in the software industry. Many engineers are finding new ways to increase productivity by offloading tasks onto LLMs, while others are finding it difficult to trust code produced artificially, even after review. Taking a look at both perspectives, this study aims to compare a human’s ability to optimize SQL queries to that of an LLM and assess the experience using both methods.

Manual query optimization is a tedious task that relies heavily on statistics, heuristics, and good intuition. The SQL developer must search for the optimal …

The Quality Assurance Machine – A Software Quality Assurance Architecture For Ml-Enabled Systems, Shane E. Downing

All-Inclusive List of Electronic Theses and Dissertations

This dissertation evaluates whether a reusable assurance architecture, the Quality Assurance Machine (QAM), can provide effective product and process quality assurance for ML-enabled software platforms. The QAM is a system-level SQA architecture that turns plans and policies into versioned configurations, executes them in controlled environments, and produces preserved run evidence that supports traceability, auditability, and controlled change. The study follows Design Science Research and evaluates the instantiated artifact using eight assurance requirements (AR1–AR8) synthesized from standards-based guidance, including IEEE 730 and ISO/IEC/IEEE 15026. A four-year longitudinal evaluation combines two methods. First, operational evidence from routine regression and release-validation runs, defect …