Software Engineering Commons^™

Boosting Symbolic Execution For Vulnerability Detection, Haoxin Tu

Dissertations and Theses Collection (Open Access)

Software systems written by humans tend to be unreliable and insecure, hence, bugs or vulnerabilities in them are inevitable. Symbolic execution has shown considerable potential in detecting diverse types of software bugs and also vulnerabilities that have severe security implications. However, existing symbolic execution engines still suffer from at least three fundamental limitations in memory modeling, path exploration, and structured input generation, which significantly impede existing engines from efficiently and effectively detecting software bugs and vulnerabilities.

The objective of this dissertation is to boost existing symbolic execution engines by designing a new memory model, two new path exploration strategies, and …

Studying Satd In Drone Systems With Human-Ai Collaboration, Leevi Rantala, Lwin Khin Shar, Mäntylä Mika V., Wei Minn, Naing Tun Yan

Research Collection School Of Computing and Information Systems

Background: Self-Admitted Technical Debt (SATD) refers to sub-optimal solutions that developers acknowledge within the source code. SATD research originated on Java projects but is expanding to other domains. We focus on SATD in drones, which are used for various critical tasks.Aims: The primary objective is to investigate SATD in drone systems. The second aim is to explore the integration of AI and human collaboration for SATD labelling and classification.Method: We conducted a sample study of SATD comments in drone systems (14 open source, 4 SDKs) to analyse the quantity and types of SATD comments present. Our study incorporates collaboration between …

Map As A By-Product: Collective Landmark Mapping From Imu Data And User-Provided Texts In Situated Tasks, Ryo Yonetani, Kotaro Hara

Research Collection School Of Computing and Information Systems

This paper presents Collective Landmark Mapper, a novel map-as-a-by-product system for generating semantic landmark maps of indoor environments. Consider users engaged in situated tasks that require them to navigate these environments and regularly take notes on their smartphones. Collective Landmark Mapper exploits the smartphone's IMU data and the user's free text input during these tasks to identify a set of landmarks encountered by the user. The identified landmarks are then aggregated across multiple users to generate a unified map representing the positions and semantic information of all landmarks. In developing the proposed system, we focused specifically on retail applications and …

Exploring Parameter-Efficient Fine-Tuning Techniques For Code Generation With Large Language Models, Martin Weyssow, Xin Zhou, Kisub Kim, David Lo, Houari A. Sahraoui

Research Collection School Of Computing and Information Systems

Large language models (LLMs) demonstrate impressive capabilities to generate accurate code snippets given natural language intents in a zero-shot manner, i.e., without the need for specific fine-tuning. While prior studies have highlighted the advantages of fine-tuning LLMs, this process incurs high computational costs, making it impractical in resource-scarce environments, particularly for models with billions of parameters. To address these challenges, previous research explored in-context learning (ICL) and retrieval-augmented generation (RAG) as strategies to guide the LLM generative process with task-specific prompt examples. However, ICL and RAG introduce inconveniences, such as the need for designing contextually relevant prompts and the absence …

Apidocbooster: An Extract-Then-Abstract Framework Leveraging Large Language Models For Augmenting Api Documentation, Chengran Yang, Jiakun Liu, Bowen Xu, Christoph Treude, Yunbo Lyu, Junda He, Ming Li, David Lo

Research Collection School Of Computing and Information Systems

API documentation is often the most trusted resource for programming. Many approaches have been proposed to augment API documentation by summarizing complementary information from external resources such as Stack Overflow. Existing extractive-based summarization approaches excel in producing faithful summaries that accurately represent the source content without input length restrictions. Nevertheless, they suffer from inherent readability limitations. On the other hand, our empirical study on the abstractive-based summarization method, i.e., GPT-4, reveals that GPT-4 can generate coherent and concise summaries but presents limitations in terms of informativeness and faithfulness. We introduce APIDocBooster, an extract-then-abstract framework that seamlessly fuses the advantages of …

Static Analysis As A Feedback Loop: Enhancing Llm-Generated Code Beyond Correctness, Scott Blyth, Sherlock Licorish, Christoph Treude, Markus Wagner

Research Collection School Of Computing and Information Systems

Large language models (LLMs) have demonstrated impressive capabilities in code generation, achieving high scores on benchmarks such as HumanEval and MBPP. However, these benchmarks primarily assess functional correctness and neglect broader dimensions of code quality, including security, reliability, readability, and maintainability. In this work, we systematically evaluate the ability of LLMs to generate high-quality code across multiple dimensions using the PythonSecurityEval benchmark. We introduce an iterative static analysis-driven prompting algorithm that leverages Bandit and Pylint to identify and resolve code quality issues. Our experiments with GPT-4o show substantial improvements: security issues reduced from >40% to 13%, readability violations from >80% …

Educator Perceptions Of Devops Teaching Recommendations And Their Alignment With Common Challenges, Marcelo Romulo Fernandes, Pablo Paiva, Samuel Lucas De Moura Ferino, Roberta Coelho, Christoph Treude, Eduardo Aranha, Uirá Kulesza

Research Collection School Of Computing and Information Systems

DevOps education presents unique pedagogical challenges due to the diversity of tools, rapid technological change, and the multidisciplinary nature of the field. Although previous work has proposed recommendations to address these challenges, it is unclear how educators perceive these recommendations and whether they align with the challenges encountered in practice. In this paper, we present a quantitative and qualitative methods study involving 11 DevOps educators who interacted with Improve, a tool that presents a curated set of educational challenges and recommendations derived from previous literature. Educators indicated which recommendations they already use, which they intend to use, and which challenges …

Improving Co-Decoding Based Security Hardening Of Code Llms Leveraging Knowledge Distillation, Dong Li, Shanfu Shu, Meng Yan, Zhongxin Liu, Chao Liu, Xiaohong Zhang, David Lo

Research Collection School Of Computing and Information Systems

Large Language Models (LLMs) have been widely adopted by developers in software development. However, the massive pretraining code data is not rigorously filtered, allowing LLMs to learn unsafe coding patterns. Several prior studies have demonstrated that code LLMs tend to generate code with potential vulnerabilities. The widespread adoption of intelligent programming assistants poses a significant threat to the software development process. Existing approaches to mitigating this risk primarily involve constructing secure data that are free of vulnerabilities and then retraining or fine-tuning the models. However, such an effort is resource intensive and requires significant manual supervision. When the model parameters …

Ponzilens+: Visualizing Bytecode Actions For Smart Ponzi Scheme Identification, Xiaolin Wen, Tai D. Nguyen, Shaolun Ruan, Qiaomu Shen, Jun Sun, Feida Zhu, Yong Wang

Research Collection School Of Computing and Information Systems

With the prevalence of smart contracts, smart Ponzi schemes have become a common fraud on blockchain and have caused significant financial loss to cryptocurrency investors in the past few years. Despite the critical importance of detecting smart Ponzi schemes, a reliable and transparent identification approach adaptive to various smart Ponzi schemes is still missing. To fill the research gap, we first extract semantic-meaningful actions to represent the execution behaviors specified in smart contract bytecodes, which are derived from a literature review and in-depth interviews with domain experts. We then propose PonziLens+, a novel visual analytic approach that provides an intuitive …

Robface: A Test Suite For Efficient Robustness Evaluation Of Face Recognition Systems, Ruihan Zhang, Jun Sun

Research Collection School Of Computing and Information Systems

Face recognition is a widely used authentication technology in practice, where robustness is required. It is thus essential to have an efficient and easy-to-use method for evaluating the robustness of (possibly third-party) trained face recognition systems. Existing approaches to evaluating the robustness of face recognition systems are either based on empirical evaluation (e.g., measuring attacking success rate using state-of-the-art attacking methods) or formal analysis (e.g., measuring the Lipschitz constant). While the former demands significant user efforts and expertise, the latter is extremely time-consuming. In pursuit of a comprehensive, efficient, easy-to-use, and scalable estimation of the robustness of face recognition systems, …

Automatic Generation Of Introductory Programming Exercises With Large Language Models, Nguyen Binh Duong Ta, Hua Gia Phuc Nguyen, Gottipati Swapna

Research Collection School Of Computing and Information Systems

Despite recent advances in code generation made possible by large language models (LLMs), programming is still an essential skill that computing students need to master now and in the foreseeable future. In learning programming, frequent practices with exercises set at an appropriate difficulty and knowledge level is of crucial importance for students. However, it’s not a trivial task for instructors to create many good quality exercises customized for each student. Programming problems found on Internet sources such as LeetCode are mostly too challenging for novice programmers with no prior coding knowledge. Recent work in AI-enabled education has been leveraging LLMs …

Finding Safety Violations Of Ai-Enabled Control Systems Through The Lens Of Synthesized Proxy Programs, Jieke Shi, Zhou Yang, Junda He, Bowen Xu, Dongsun Kim, Donggyun Han, David Lo

Research Collection School Of Computing and Information Systems

Given the increasing adoption of modern AI-enabled control systems, ensuring their safety and reliability has become a critical task in software testing. One prevalent approach to testing control systems is falsification, which aims to find an input signal that causes the control system to violate a formal safety specification using optimization algorithms. However, applying falsification to AI-enabled control systems poses two significant challenges: (1) it requires the system to execute numerous candidate test inputs, which can be time-consuming, particularly for systems with AI models that have many parameters, and (2) multiple safety requirements are typically defined as a conjunctive specification, …

From Release To Adoption: Challenges In Reusing Pre-Trained Ai Models For Downstream Developers, Peerachai Banyongrakkul, Mansooreh Zahedi, Patanamon Thongtanunam, Christoph Treude, Haoyu Gao

Research Collection School Of Computing and Information Systems

Pre-trained models (PTMs) have gained widespread popularity and achieved remarkable success across various fields, driven by their groundbreaking performance and easy accessibility through hosting providers. However, the challenges faced by downstream developers in reusing PTMs in software systems are less explored. To bridge this knowledge gap, we qualitatively created and analyzed a dataset of 840 PTM-related issue reports from 31 OSS GitHub projects. We systematically developed a comprehensive taxonomy of PTM-related challenges that developers face in downstream projects. Our study identifies seven key categories of challenges that downstream developers face in reusing PTMs, such as model usage, model performance, and …

Stylegan-∞: Extending Stylegan To Arbitrary-Ratio Translation With Stylebook, Yihua Dai, Tianyi Xiang, Bailin Deng, Yong Du, Hongmin Cai, Jing Qin, Shengfeng He

Research Collection School Of Computing and Information Systems

Although pre-trained large-scale generative models StyleGAN series have proven to be effective in various editing and translation tasks, they are limited to pre-defined fixed aspect ratio. To overcome this limitation, we propose StyleGAN-∞, a model that enables pre-trained StyleGAN to perform arbitrary-ratio conditional synthesis. Our key insight is to distill the expressive StyleGAN features into a StyleBook, such that an arbitrary-ratio condition can be translated to other forms by properly assembling pre-defined StyleBook vectors. To learn and leverage the StyleBook, we employ a network with three distinct stages, each corresponding to StyleBook extraction, StyleBook correspondence learning, and arbitrary-ratio synthesis. Extensive …

The Integration Of Agile Methodologies In Devops Practices Within The Information Technology Industry, Ashley Hourigan, Ridewaan Hanslo

African Conference on Information Systems and Technology

The demand for rapid software delivery in the Information Technology (IT) industry has significantly intensified, emphasising the need for faster software products and service releases with enhanced features to meet customer expectations. Agile methodologies are replacing traditional approaches such as Waterfall, where flexibility, iterative development and adaptation to change are favoured over rigid planning and execution. DevOps, a subsequent evolution from Agile, emphasises collaborative efforts in development and operations teams, focusing on continuous integration and deployment to deliver resilient and high-quality software products and services. This study aims to critically assess both Agile and DevOps practices in the IT industry …

Designing A Data Collection And Visualization Toolkit For Scalable Tensor Algebra In Quantum Chemistry Applications, Epiya J. Ebiapia

LSU Master's Theses

Large-scale quantum chemistry computations, such as those executed with the Tensor Algebra for Many-body Methods (TAMM) framework, require careful configuration of runtime parameters to achieve high performance and cost efficiency in high-performance computing (HPC) and cloud environments. Without effective performance analysis tools, researchers risk inefficient use of computational resources, leading to longer runtimes and higher costs.

To address this challenge, this thesis presents the design and implementation of a performance profiling and visualization toolkit for TAMM, developed as part of the DOE TEC4 project in collaboration with Pacific Northwest National Laboratory, Microsoft, and Louisiana State University. The toolkit collects detailed …

Freqllm: Frequency-Aware Large Language Models For Time Series Forecasting, Shunan Wang, Min Gao, Zongwei Wang, Yibing Bai, Feng Jiang, Guansong Pang

Research Collection School Of Computing and Information Systems

Large Language Models (LLMs) have recently shown promise in Time Series Forecasting (TSF) by effectively capturing intricate time-domain dependencies. However, our preliminary experiments reveal that standard LLM-based approaches often fail to capture global correlations, limiting predictive performance. We found that embedding frequency-domain signals smooths weight distributions and enhances structured correlations by clearly separating global trends (low-frequency components) from local variations (high-frequency components). Building on these insights, we propose FreqLLM, a novel framework that integrates frequency-domain semantic alignment into LLMs to refine prompts for improved time series analysis. By bridging the gap between frequency signals and textual embeddings, FreqLLM effectively captures …

Preface, Special Issue For The 16th International Conference On Graph Transformation (Icgt 2023), Maribel Fernández, Christopher M. Poskitt

Research Collection School Of Computing and Information Systems

This special issue contains six extended versions of papers presented at the 16th International Conference on Graph Transformation (ICGT 2023), held in Leicester, UK, on 19–20 July 2023. The conference was part of STAF 2023 (Software Technologies: Applications and Foundations) and was held under the auspices of the European Association for Theoretical Computer Science (EATCS), the European Association of Software Science and Technology (EASST), and the IFIP Working Group 1.3, Foundations of Systems Specification.

Evaluating And Mitigating Linguistic Discrimination In Large Language Models: Perspectives On Safety Equity And Knowledge Equity, Guoliang Dong, Haoyu Wang, Jun Sun, Xinyu Wang

Research Collection School Of Computing and Information Systems

By training on text in various languages, large language models (LLMs) typically possess multilingual support and demonstrate remarkable capabilities in solving tasks described in different languages. However, LLMs can exhibit linguistic discrimination due to the uneven distribution of training data across languages. That is, LLMs are hard to keep the consistency of responses when faced with the same task but depicted in different languages. In this study, we first explore the consistency in the LLMs’ outputs responding to queries in various languages from two aspects: safety and quality. We conduct this analysis with two datasets (AdvBench and NQ) based on …

Assessing The Robustness Of Test Selection Methods For Deep Neural Networks, Qiang Hu, Yuejun Guo, Xiaofei Xie, Maxime Cordy, Wei Ma, Mike Papadakis, Lei Ma, Yves Le Traon

Research Collection School Of Computing and Information Systems

Regularly testing deep learning-powered systems on newly collected data is critical to ensure their reliability, robustness, and efficacy in real-world applications. This process is demanding due to the significant time and human effort required for labeling new data. While test selection methods alleviate manual labor by labeling and evaluating only a subset of data while meeting testing criteria, we observe that such methods with reported promising results are simply evaluated, e.g., testing on original test data. The question arises: are they always reliable? In this article, we explore when and to what extent test selection methods fail. First, we identify …