Stealthy Backdoor Attack For Code Models,
2024
Singapore Management University
Stealthy Backdoor Attack For Code Models, Zhou Yang, Bowen Xu, Jie M. Zhang, Hong Jin Kang, Jieke Shi, Junda He, David Lo
Research Collection School Of Computing and Information Systems
Code models, such as CodeBERT and CodeT5, offer general-purpose representations of code and play a vital role in supporting downstream automated software engineering tasks. Most recently, code models were revealed to be vulnerable to backdoor attacks. A code model that is backdoor-attacked can behave normally on clean examples but will produce pre-defined malicious outputs on examples injected with that activate the backdoors. Existing backdoor attacks on code models use unstealthy and easy-to-detect triggers. This paper aims to investigate the vulnerability of code models with backdoor attacks. To this end, we propose A (dversarial eature as daptive Back). A achieves stealthiness …
Teaching Software Development For Real-World Problems Using A Microservice-Based Collaborative Problem-Solving Approach,
2024
Singapore Management University
Teaching Software Development For Real-World Problems Using A Microservice-Based Collaborative Problem-Solving Approach, Yi Meng Lau, Christian Michael Koh, Lingxiao Jiang
Research Collection School Of Computing and Information Systems
Experienced and skillful software developers are needed in organizations to develop software products effective for their business with shortened time-to-market. Such developers will not only need to code but also be able to work in teams and collaboratively solve real-world problems that organizations arefacing. It is challenging for educators to nurture students to become such developers with strong technical, social, and cognitive skills. Towards addressing the challenge, this study presents a Collaborative Software Development Project Framework for a course that focuses on learning microservices architectures anddeveloping a software application for a real-world business. Students get to work in teams to …
W4-Groups: Modeling The Who, What, When And Where Of Group Behavior Via Mobility Sensing,
2024
Singapore Management University
W4-Groups: Modeling The Who, What, When And Where Of Group Behavior Via Mobility Sensing, Akansha Atrey, Camellia Zakaria, Rajesh Krishna Balan, Prashant Shenoy
Research Collection School Of Computing and Information Systems
Human social interactions occur in group settings of varying sizes and locations, depending on the type of social activity. The ability to distinguish group formations based on their purposes transforms how group detection mechanisms function. Not only should such tools support the effective detection of serendipitous encounters, but they can derive categories of relation types among users. Determining who is involved, what activity is performed, and when and where the activity occurs are critical to understanding group processes in greater depth, including supporting goal-oriented applications (e.g., performance, productivity, and mental health) that require sensing social factors. In this work, we …
Encoding Version History Context For Better Code Representation,
2024
Singapore Management University
Encoding Version History Context For Better Code Representation, Huy Nguyen, Christoph Treude, Patanamon Thongtanunam
Research Collection School Of Computing and Information Systems
With the exponential growth of AI tools that generate source code, understanding software has become crucial. When developers comprehend a program, they may refer to additional contexts to look for information, e.g. program documentation or historical code versions. Therefore, we argue that encoding this additional contextual information could also benefit code representation for deep learning. Recent papers incorporate contextual data (e.g. call hierarchy) into vector representation to address program comprehension problems. This motivates further studies to explore additional contexts, such as version history, to enhance models' understanding of programs. That is, insights from version history enable recognition of patterns in …
Exploring The Potential Of Chatgpt In Automated Code Refinement: An Empirical Study,
2024
Singapore Management University
Exploring The Potential Of Chatgpt In Automated Code Refinement: An Empirical Study, Qi Guo, Shangqing Liu, Junming Cao, Xiaohong Li, Xin Peng, Xiaofei Xie, Bihuan Chen
Research Collection School Of Computing and Information Systems
Code review is an essential activity for ensuring the quality and maintainability of software projects. However, it is a time-consuming and often error-prone task that can significantly impact the development process. Recently, ChatGPT, a cutting-edge language model, has demonstrated impressive performance in various natural language processing tasks, suggesting its potential to automate code review processes. However, it is still unclear how well ChatGPT performs in code review tasks. To fill this gap, in this paper, we conduct the first empirical study to understand the capabilities of ChatGPT in code review tasks, specifically focusing on automated code refinement based on given …
My Github Sponsors Profile Is Live!": Investigating The Impact Of Twitter/X Mentions On Github Sponsors,
2024
Singapore Management University
My Github Sponsors Profile Is Live!": Investigating The Impact Of Twitter/X Mentions On Github Sponsors, Youmei Fan, Tao Xiao, Hideaki Hata, Christoph Treude, Kenichi Matsumoto
Research Collection School Of Computing and Information Systems
GitHub Sponsors was launched in 2019, enabling donations to opensource software developers to provide financial support, as per GitHub’s slogan: “Invest in the projects you depend on”. However, a 2022 study on GitHub Sponsors found that only two-fifths of developers who were seeking sponsorship received a donation. The study found that, other than internal actions (such as offering perks to sponsors), developers had advertised their GitHub Sponsors profiles on social media, such as Twitter (also known as X). Therefore, in this work, we investigate the impact of tweets that contain links to GitHub Sponsors profiles on sponsorship, as well as …
Test Optimization In Dnn Testing: A Survey,
2024
Singapore Management University
Test Optimization In Dnn Testing: A Survey, Qiang Hu, Yuejun Guo, Xiaofei Xie, Maxime Cordy, Lei Ma, Mike Papadakis, Yves Le Traon
Research Collection School Of Computing and Information Systems
This article presents a comprehensive survey on test optimization in deep neural network (DNN) testing. Here, test optimization refers to testing with low data labeling effort. We analyzed 90 papers, including 43 from the software engineering (SE) community, 32 from the machine learning (ML) community, and 15 from other communities. Our study: (i) unifies the problems as well as terminologies associated with low-labeling cost testing, (ii) compares the distinct focal points of SE and ML communities, and (iii) reveals the pitfalls in existing literature. Furthermore, we highlight the research opportunities in this domain.
Unveiling Memorization In Code Models,
2024
Singapore Management University
Unveiling Memorization In Code Models, Zhou Yang, Zhipeng Zhao, Chenyu Wang, Jieke Shi, Dongsun Kim, Donggyun Han, David Lo
Research Collection School Of Computing and Information Systems
The availability of large-scale datasets, advanced architectures, and powerful computational resources have led to effective code models that automate diverse software engineering activities. The datasets usually consist of billions of lines of code from both open-source and private repositories. A code model memorizes and produces source code verbatim, which potentially contains vulnerabilities, sensitive information, or code with strict licenses, leading to potential security and privacy issues.This paper investigates an important problem: to what extent do code models memorize their training data? We conduct an empirical study to explore memorization in large pre-trained code models. Our study highlights that simply extracting …
Acav: A Framework For Automatic Causality Analysis In Autonomous Vehicle Accident Recordings,
2024
Singapore Management University
Acav: A Framework For Automatic Causality Analysis In Autonomous Vehicle Accident Recordings, Huijia Sun, Christopher M. Poskitt, Yang Sun, Jun Sun, Yuqi Chen
Research Collection School Of Computing and Information Systems
The rapid progress of autonomous vehicles (AVs) has brought the prospect of a driverless future closer than ever. Recent fatalities, however, have emphasized the importance of safety validation through large-scale testing. Multiple approaches achieve this fully automatically using high-fidelity simulators, i.e., by generating diverse driving scenarios and evaluating autonomous driving systems (ADSs) against different test oracles. While effective at finding violations, these approaches do not identify the decisions and actions that caused them -- information that is critical for improving the safety of ADSs. To address this challenge, we propose ACAV, an automated framework designed to conduct causality analysis for …
Experience Report: Identifying Common Misconceptions And Errors Of Novice Programmers With Chatgpt,
2024
Singapore Management University
Experience Report: Identifying Common Misconceptions And Errors Of Novice Programmers With Chatgpt, Hua Leong Fwa
Research Collection School Of Computing and Information Systems
Identifying the misconceptions of novice programmers is pertinent for informing instructors of the challenges faced by their students in learning computer programming. In the current literature, custom tools, test scripts were developed and, in most cases, manual effort to go through the individual codes were required to identify and categorize the errors latent within the students' code submissions. This entails investment of substantial effort and time from the instructors. In this study, we thus propose the use of ChatGPT in identifying and categorizing the errors. Using prompts that were seeded only with the student's code and the model code solution …
Improving Automated Code Reviews: Learning From Experience,
2024
University of Melbourne
Improving Automated Code Reviews: Learning From Experience, Hong Yi Lin, Patanamon Thongtanunam, Christoph Treude, Wachiraphan Charoenwet
Research Collection School Of Computing and Information Systems
Modern code review is a critical quality assurance process that is widely adopted in both industry and open source software environments. This process can help newcomers learn from the feedback of experienced reviewers; however, it often brings a large workload and stress to reviewers. To alleviate this burden, the field of automated code reviews aims to automate the process, teaching large language models to provide reviews on submitted code, just as a human would. A recent approach pre-trained and fine-tuned the code intelligent language model on a large-scale code review corpus. However, such techniques did not fully utilise quality reviews …
Dronlomaly: Runtime Log-Based Anomaly Detector For Dji Drones,
2024
Singapore Management University
Dronlomaly: Runtime Log-Based Anomaly Detector For Dji Drones, Wei Minn, Naing Tun Yan, Lwin Khin Shar, Lingxiao Jiang
Research Collection School Of Computing and Information Systems
We present an automated tool for realtime detection of anomalous behaviors while a DJI drone is executing a flight mission. The tool takes sensor data logged by drone at fixed time intervals and performs anomaly detection using a Bi-LSTM model. The model is trained on baseline flight logs from a successful mission physically or via a simulator. The tool has two modules --- the first module is responsible for sending the log data to the remote controller station, and the second module is run as a service in the remote controller station powered by a Bi-LSTM model, which receives the …
The Impact Of Bug Localization Based On Crash Report Mining: A Developers' Perspective,
2024
Singapore Management University
The Impact Of Bug Localization Based On Crash Report Mining: A Developers' Perspective, Marcos Medeiros, Uirá Kulesza, Roberta Coelho, Rodrigo Bonifacio, Christoph Treude, Eiji Adachi Barbosa
Research Collection School Of Computing and Information Systems
Developers often use crash reports to understand the root cause of bugs. However, locating the buggy source code snippet from such information is a challenging task, mainly when the log database contains many crash reports. To mitigate this issue, recent research has proposed and evaluated approaches for grouping crash report data and using stack trace information to locate bugs. The effectiveness of such approaches has been evaluated by mainly comparing the candidate buggy code snippets with the actual changed code in bug-fix commits—which happens in the context of retrospective repository mining studies. Therefore, the existing literature still lacks discussing the …
Githubinclusifier: Finding And Fixing Non-Inclusive Language In Github Repositories,
2024
Singapore Management University
Githubinclusifier: Finding And Fixing Non-Inclusive Language In Github Repositories, Liam Todd, John Grundy, Christoph Treude
Research Collection School Of Computing and Information Systems
Non-inclusive language in software artefacts has been recognised as a serious problem. We describe a tool to find and fix non-inclusive language in a variety of GitHub repository artefacts. These include various README files, PDFs, code comments, and code. A wide variety of non-inclusive language including racist, ageist, ableist, violent and others are located and issues created, tagging the artefacts for checking. Suggested fixes can be generated using third-party LLM APIs, and approved changes made to documents, including code refactorings, and committed to the repository. The tool and evaluation data are available from: https://github. com/LiamTodd/github-inclusifier
Classifying Source Code: How Far Can Compressor-Based Classifiers Go?,
2024
Singapore Management University
Classifying Source Code: How Far Can Compressor-Based Classifiers Go?, Zhou Yang
Research Collection School Of Computing and Information Systems
Pre-trained language models of code, which are built upon large-scale datasets, millions of trainable parameters, and high computational resources cost, have achieved phenomenal success. Recently, researchers have proposed a compressor-based classifier (Cbc); it trains no parameters but is found to outperform BERT. We conduct the first empirical study to explore whether this lightweight alternative can accurately classify source code. Our study is more than applying Cbc to code-related tasks. We first identify an issue that the original implementation overestimates Cbc. After correction, Cbc's performance on defect prediction drops from 80.7% to 63.0%, which is still comparable to CodeBERT (63.7%). We …
Creative And Correct: Requesting Diverse Code Solutions From Ai,
2024
Monash University
Creative And Correct: Requesting Diverse Code Solutions From Ai, Scott Blyth, Markus Wagner, Christoph Treude
Research Collection School Of Computing and Information Systems
AI foundation models have the capability to produce a wide array of responses to a single prompt, a feature that is highly beneficial in software engineering to generate diverse code solutions. However, this advantage introduces a significant trade-off between diversity and correctness. In software engineering tasks, diversity is key to exploring design spaces and fostering creativity, but the practical value of these solutions is heavily dependent on their correctness. Our study systematically investigates this trade-off using experiments with HumanEval tasks, exploring various parameter settings and prompting strategies. We assess the diversity of code solutions using similarity metrics from the code …
Large Language Model For Vulnerability Detection: Emerging Results And Future Directions,
2024
Singapore Management University
Large Language Model For Vulnerability Detection: Emerging Results And Future Directions, Xin Zhou, Ting Zhang, David Lo
Research Collection School Of Computing and Information Systems
Previous learning-based vulnerability detection methods relied on either medium-sized pre-trained models or smaller neural networks from scratch. Recent advancements in Large Pre-Trained Language Models (LLMs) have showcased remarkable few-shot learning capabilities in various tasks. However, the effectiveness of LLMs in detecting software vulnerabilities is largely unexplored. This paper aims to bridge this gap by exploring how LLMs perform with various prompts, particularly focusing on two state-of-the-art LLMs: GPT-3.5 and GPT-4. Our experimental results showed that GPT-3.5 achieves competitive performance with the prior state-of-the-art vulnerability detection approach and GPT-4 consistently outperformed the state-of-the-art.
Code Search Is All You Need? Improving Code Suggestions With Code Search,
2024
Singapore Management University
Code Search Is All You Need? Improving Code Suggestions With Code Search, Junkai Chen, Xing Hu, Zhenhao Li, Cuiyun Gao, Xin Xia, David Lo
Research Collection School Of Computing and Information Systems
Modern integrated development environments (IDEs) provide various automated code suggestion techniques (e.g., code completion and code generation) to help developers improve their efficiency. Such techniques may retrieve similar code snippets from the code base or leverage deep learning models to provide code suggestions. However, how to effectively enhance the code suggestions using code retrieval has not been systematically investigated. In this paper, we study and explore a retrieval-augmented framework for code suggestions. Specifically, our framework leverages different retrieval approaches and search strategies to search similar code snippets. Then the retrieved code is used to further enhance the performance of language …
Greening Large Language Models Of Code,
2024
Singapore Management University
Greening Large Language Models Of Code, Jieke Shi, Zhou Yang, Hong Jin Kang, Bowen Xu, Junda He, David Lo
Research Collection School Of Computing and Information Systems
Large language models of code have shown remarkable effectiveness across various software engineering tasks. Despite the availability of many cloud services built upon these powerful models, there remain several scenarios where developers cannot take full advantage of them, stemming from factors such as restricted or unreliable internet access, institutional privacy policies that prohibit external transmission of code to third-party vendors, and more. Therefore, developing a compact, efficient, and yet energy-saving model for deployment on developers' devices becomes essential.To this aim, we propose Avatar, a novel approach that crafts a deployable model from a large language model of code by optimizing …
Ps3: Precise Patch Presence Test Based On Semantic Symbolic Signature,
2024
Singapore Management University
Ps3: Precise Patch Presence Test Based On Semantic Symbolic Signature, Qi Zhan, Xing Hu, Zhiyang Li, Xin Xia, David Lo, Shanping Li
Research Collection School Of Computing and Information Systems
During software development, vulnerabilities have posed a significant threat to users. Patches are the most effective way to combat vulnerabilities. In a large-scale software system, testing the presence of a security patch in every affected binary is crucial to ensure system security. Identifying whether a binary has been patched for a known vulnerability is challenging, as there may only be small differences between patched and vulnerable versions. Existing approaches mainly focus on detecting patches that are compiled in the same compiler options. However, it is common for developers to compile programs with very different compiler options in different situations, which …
