Open Access. Powered by Scholars. Published by Universities.®

Software Engineering Commons™

Open Access. Powered by Scholars. Published by Universities.®

Singapore Management University

Discipline
Keyword
Publication Year
Publication
Publication Type

Articles 391 - 420 of 2211

Full-Text Articles in Software Engineering

Evlog: Identifying Anomalous Logs Over Software Evolution, Yintong Huo, Cheryl Lee, Yuxin Su, Shiwen Shan, Jinyang Liu, Michael Lyu Oct 2023

Evlog: Identifying Anomalous Logs Over Software Evolution, Yintong Huo, Cheryl Lee, Yuxin Su, Shiwen Shan, Jinyang Liu, Michael Lyu

Research Collection School Of Computing and Information Systems

Software logs record system activities, aiding maintainers in identifying the underlying causes for failures and enabling prompt mitigation actions. However, maintainers need to inspect a large volume of daily logs to identify the anomalous logs that reveal failure details for further diagnosis. Thus, how to automatically distinguish these anomalous logs from normal logs becomes a critical problem. Existing approaches alleviate the burden on software maintainers, but they are built upon an improper yet critical assumption: logging statements in the software remain unchanged. While software keeps evolving, our empirical study finds that evolving software brings three challenges: log parsing errors, evolving …

Go to article

Visilience: An Interactive Visualization Framework For Resilience Analysis Using Control-Flow Graph, Hailong Jiang, Shaolun Ruan, Bo Fang, Yong Wang, Qiang Guan Oct 2023

Visilience: An Interactive Visualization Framework For Resilience Analysis Using Control-Flow Graph, Hailong Jiang, Shaolun Ruan, Bo Fang, Yong Wang, Qiang Guan

Research Collection School Of Computing and Information Systems

Soft errors have become one of the main concerns for the resilience of HPC applications, as these errors can cause HPC applications to generate serious outcomes such as silent data corruption (SDC). Many approaches have been proposed to analyze the resilience of HPC applications. However, existing studies rarely address the challenges of analysis result perception. Specifically, resilience analysis techniques often produce a massive volume of unstructured data, making it difficult for programmers to perform resilience analysis due to non-intuitive raw data. Furthermore, different analysis models produce diverse results with multiple levels of detail, which can create obstacles to compare and …

Go to article

Towards An Effective And Interpretable Refinement Approach For Dnn Verification, Jiaying Li, Guangdong Bai, Long H. Pham, Jun Sun Oct 2023

Towards An Effective And Interpretable Refinement Approach For Dnn Verification, Jiaying Li, Guangdong Bai, Long H. Pham, Jun Sun

Research Collection School Of Computing and Information Systems

Recently, several abstraction refinement techniques have been proposed to improve the verification precision for deep neural networks (DNNs). However, these techniques usually take many refinement steps to verify a property and the refinement decision in each step is hard to interpret, thus hindering their analysis, reasoning and optimization.In this work, we propose SURGEON, a novel DNN verification refinement approach that is both effective and interpretable, allowing analyst to understand why and how each refinement decision is made. The main insight is to leverage the ‘interpretable’ nature of debugging processes and formulate the verification refinement problem as a debugging problem. Given …

Go to article

Visually Analyzing Company-Wide Software Service Dependencies: An Industrial Case Study, Sebastian Baltes, Brian Pfitzmann, Thomas Kowark, Christoph Treude, Fabian Beck Oct 2023

Visually Analyzing Company-Wide Software Service Dependencies: An Industrial Case Study, Sebastian Baltes, Brian Pfitzmann, Thomas Kowark, Christoph Treude, Fabian Beck

Research Collection School Of Computing and Information Systems

Managing dependencies between software services is a crucial task for any company operating cloud applications. Visualizations can help to understand and maintain these com-plex dependencies. In this paper, we present a force-directed service dependency visualization and filtering tool that has been developed and used within SAP. The tool's use cases include guiding service retirement as well as understanding service deployment landscapes and their relationship to the company's organizational structure. We report how we built and adapted the tool under strict time constraints to address the requirements of our users. We further share insights on how we enabled internal adoption. For …

Go to article

Boosting Adversarial Training In Safety-Critical Systems Through Boundary Data Selection, Yifan Jia, Christopher M. Poskitt, Peixin Zhang, Jingyi Wang, Jun Sun, Sudipta Chattopadhyay Oct 2023

Boosting Adversarial Training In Safety-Critical Systems Through Boundary Data Selection, Yifan Jia, Christopher M. Poskitt, Peixin Zhang, Jingyi Wang, Jun Sun, Sudipta Chattopadhyay

Research Collection School Of Computing and Information Systems

AI-enabled collaborative robots are designed to be used in close collaboration with humans, thus requiring stringent safety standards and quick response times. Adversarial attacks pose a significant threat to the deep learning models of these systems, making it crucial to develop methods to improve the models' robustness against them. Adversarial training is one approach to improve their robustness: it works by augmenting the training data with adversarial examples. This, unfortunately, comes with the cost of increased computational overhead and extended training times. In this work, we balance the need for additional adversarial data with the goal of minimizing the training …

Go to article

K-St: A Formal Executable Semantics Of The Structured Text Language For Plcs, Kun Wang, Jingyi Wang, Christopher M. Poskitt, Xiangxiang Chen, Jun Sun, Peng Cheng Sep 2023

K-St: A Formal Executable Semantics Of The Structured Text Language For Plcs, Kun Wang, Jingyi Wang, Christopher M. Poskitt, Xiangxiang Chen, Jun Sun, Peng Cheng

Research Collection School Of Computing and Information Systems

Programmable Logic Controllers (PLCs) are responsible for automating process control in many industrial systems (e.g. in manufacturing and public infrastructure), and thus it is critical to ensure that they operate correctly and safely. The majority of PLCs are programmed in languages such as Structured Text (ST). However, a lack of formal semantics makes it difficult to ascertain the correctness of their translators and compilers, which vary from vendor-to-vendor. In this work, we develop K-ST, a formal executable semantics for ST in the K framework. Defined with respect to the IEC 61131-3 standard and PLC vendor manuals, K-ST is a high-level …

Go to article

Generative Model-Based Testing On Decision-Making Policies, Zhuo Li, Xiongfei Wu, Derui Zhu, Mingfei Cheng, Siyuan Chen, Fuyuan Zhang, Xiaofei Xie, Lei Ma, Jianjun Zhao Sep 2023

Generative Model-Based Testing On Decision-Making Policies, Zhuo Li, Xiongfei Wu, Derui Zhu, Mingfei Cheng, Siyuan Chen, Fuyuan Zhang, Xiaofei Xie, Lei Ma, Jianjun Zhao

Research Collection School Of Computing and Information Systems

The reliability of decision-making policies is urgently important today as they have established the fundamentals of many critical applications, such as autonomous driving and robotics. To ensure reliability, there have been a number of research efforts on testing decision-making policies that solve Markov decision processes (MDPs). However, due to the deep neural network (DNN)-based inherit and infinite state space, developing scalable and effective testing frameworks for decision-making policies still remains open and challenging.In this paper, we present an effective testing framework for decision-making policies. The framework adopts a generative diffusion model-based test case generator that can easily adapt to different …

Go to article

Testsgd: Interpretable Testing Of Neural Networks Against Subtle Group Discrimination, Mengdi Zhang, Jun Sun, Jingyi Wang, Bing Sun Sep 2023

Testsgd: Interpretable Testing Of Neural Networks Against Subtle Group Discrimination, Mengdi Zhang, Jun Sun, Jingyi Wang, Bing Sun

Research Collection School Of Computing and Information Systems

Discrimination has been shown in many machine learning applications, which calls for sufficient fairness testing before their deployment in ethic-relevant domains. One widely concerning type of discrimination, testing against group discrimination, mostly hidden, is much less studied, compared with identifying individual discrimination. In this work, we propose TestSGD, an interpretable testing approach which systematically identifies and measures hidden (which we call ‘subtle’) group discrimination of a neural network characterized by conditions over combinations of the sensitive attributes. Specifically, given a neural network, TestSGD first automatically generates an interpretable rule set which categorizes the input space into two groups. Alongside, TestSGD …

Go to article

Experimental Comparison Of Features, Analyses, And Classifiers For Android Malware Detection, Lwin Khin Shar, Biniam Fisseha Demissie, Mariano Ceccato, Naing Tun Yan, David Lo, Lingxiao Jiang, Christoph Bienert Sep 2023

Experimental Comparison Of Features, Analyses, And Classifiers For Android Malware Detection, Lwin Khin Shar, Biniam Fisseha Demissie, Mariano Ceccato, Naing Tun Yan, David Lo, Lingxiao Jiang, Christoph Bienert

Research Collection School Of Computing and Information Systems

Android malware detection has been an active area of research. In the past decade, several machine learning-based approaches based on different types of features that may characterize Android malware behaviors have been proposed. The usually-analyzed features include API usages and sequences at various abstraction levels (e.g., class and package), extracted using static or dynamic analysis. Additionally, features that characterize permission uses, native API calls and reflection have also been analyzed. Initial works used conventional classifiers such as Random Forest to learn on those features. In recent years, deep learning-based classifiers such as Recurrent Neural Network have been explored. Considering various …

Go to article

Automated Question Title Reformulation By Mining Modifcation Logs From Stack Overflow, Ke Liu, Xiang Chen, Chunyang Chen, Xiaofei Xie, Zhanqi Cui Sep 2023

Automated Question Title Reformulation By Mining Modifcation Logs From Stack Overflow, Ke Liu, Xiang Chen, Chunyang Chen, Xiaofei Xie, Zhanqi Cui

Research Collection School Of Computing and Information Systems

In Stack Overflow, developers may not clarify and summarize the critical problems in the question titles due to a lack of domain knowledge or poor writing skills. Previous studies mainly focused on automatically generating the question titles by analyzing the posts’ problem descriptions and code snippets. In this study, we aim to improve title quality from the perspective of question title reformulation and propose a novel approach QETRA motivated by the findings of our formative study. Specifically, by mining modification logs from Stack Overflow, we first extract title reformulation pairs containing the original title and the reformulated title. Then we …

Go to article

Endwatch: A Practical Method For Detecting Non-Termination In Real-World Software, Yao Zhang, Xiaofei Xie, Yi Li, Sen Chen, Cen Zhang, Xiaohong Li Sep 2023

Endwatch: A Practical Method For Detecting Non-Termination In Real-World Software, Yao Zhang, Xiaofei Xie, Yi Li, Sen Chen, Cen Zhang, Xiaohong Li

Research Collection School Of Computing and Information Systems

Detecting non-termination is crucial for ensuring program correctness and security, such as preventing denial-of-service attacks. While termination analysis has been studied for many years, existing methods have limited scalability and are only effective on small programs. To address this issue, we propose a practical termination checking technique, called EndWatch, for detecting non-termination through testing. Specifically, we introduce two methods to generate non-termination oracles based on checking state revisits, i.e., if the program returns to a previously visited state at the same program location, it does not terminate. The non-termination oracles can be incorporated into testing tools (e.g., AFL used in …

Go to article

Edge Distraction-Aware Salient Object Detection, Sucheng Ren, Wenxi Liu, Jianbo Jiao, Guoqiang Han, Shengfeng He Sep 2023

Edge Distraction-Aware Salient Object Detection, Sucheng Ren, Wenxi Liu, Jianbo Jiao, Guoqiang Han, Shengfeng He

Research Collection School Of Computing and Information Systems

Integrating low-level edge features has been proven to be effective in preserving clear boundaries of salient objects. However, the locality of edge features makes it difficult to capture globally salient edges, leading to distraction in the final predictions. To address this problem, we propose to produce distraction-free edge features by incorporating cross-scale holistic interdependencies between high-level features. In particular, we first formulate our edge features extraction process as a boundary-filling problem. In this way, we enforce edge features to focus on closed boundaries instead of those disconnected background edges. Second, we propose to explore cross-scale holistic contextual connections between every …

Go to article

Revisiting The Identification Of The Co-Evolution Of Production And Test Code, Weifeng Sun, Meng Yan, Zhongxin Liu, Xin Xia, Yan Lei, David Lo Sep 2023

Revisiting The Identification Of The Co-Evolution Of Production And Test Code, Weifeng Sun, Meng Yan, Zhongxin Liu, Xin Xia, Yan Lei, David Lo

Research Collection School Of Computing and Information Systems

Many software processes advocate that the test code should co-evolve with the production code. Prior work usually studies such co-evolution based on production-test co-evolution samples mined from software repositories. A production-test co-evolution sample refers to a pair of a test code change and a production code change where the test code change triggers or is triggered by the production code change. The quality of the mined samples is critical to the reliability of research conclusions. Existing studies mined production-test co-evolution samples based on the following assumption: if a test class and its associated production class change together in one commit, …

Go to article

Autoconf: Automated Configuration Of Unsupervised Learning Systems Using Metamorphic Testing And Bayesian Optimization, Lwin Khin Shar, Goknil Arda, Erik Johannes Husom, Sagar Sen Sen, Naing Tun Yan, Kisub Kim Sep 2023

Autoconf: Automated Configuration Of Unsupervised Learning Systems Using Metamorphic Testing And Bayesian Optimization, Lwin Khin Shar, Goknil Arda, Erik Johannes Husom, Sagar Sen Sen, Naing Tun Yan, Kisub Kim

Research Collection School Of Computing and Information Systems

Unsupervised learning systems using clustering have gained significant attention for numerous applications due to their unique ability to discover patterns and structures in large unlabeled datasets. However, their effectiveness highly depends on their configuration, which requires domain-specific expertise and often involves numerous manual trials. Specifically, selecting appropriate algorithms and hyperparameters adds to the com- plexity of the configuration process. In this paper, we propose, apply, and assess an automated approach (AutoConf) for config- uring unsupervised learning systems using clustering, leveraging metamorphic testing and Bayesian optimization. Metamorphic testing is utilized to verify the configurations of unsupervised learning systems by applying a …

Go to article

Log-Based Anomaly Detection Based On Evt Theory With Feedback, Jinyang Liu, Junjie Huang, Yintong Huo, Zhihan Jiang, Jiazhen Gu, Zhuangbin Chen, Cong Feng, Minzhi Yan, R. Michael Lyu Sep 2023

Log-Based Anomaly Detection Based On Evt Theory With Feedback, Jinyang Liu, Junjie Huang, Yintong Huo, Zhihan Jiang, Jiazhen Gu, Zhuangbin Chen, Cong Feng, Minzhi Yan, R. Michael Lyu

Research Collection School Of Computing and Information Systems

System logs play a critical role in maintaining the reliability of software systems. Fruitful studies have explored automatic log-based anomaly detection and achieved notable accuracy on benchmark datasets. However, when applied to large-scale cloud systems, these solutions face limitations due to high resource consumption and lack of adaptability to evolving logs. In this paper, we present an accurate, lightweight, and adaptive log-based anomaly detection framework, referred to as SeaLog. Our method introduces a Trie-based Detection Agent (TDA) that employs a lightweight, dynamically-growing trie structure for real-time anomaly detection. To enhance TDA's accuracy in response to evolving log data, we enable …

Go to article

Are We Ready To Embrace Generative Ai For Software Q&A?, Bowen Xu, Thanh-Dat Nguyen, Thanh Le Cong, Thong Hoang, Jiakun Liu, Kisub Kim, Chen Gong, Changan Niu, Chenyu Wang, Xuan-Bach Dinh Le, David Lo Sep 2023

Are We Ready To Embrace Generative Ai For Software Q&A?, Bowen Xu, Thanh-Dat Nguyen, Thanh Le Cong, Thong Hoang, Jiakun Liu, Kisub Kim, Chen Gong, Changan Niu, Chenyu Wang, Xuan-Bach Dinh Le, David Lo

Research Collection School Of Computing and Information Systems

Stack Overflow, the world's largest software Q&A (SQA) website, is facing a significant traffic drop due to the emergence of generative AI techniques. ChatGPT is banned by Stack Overflow after only 6 days from its release. The main reason provided by the official Stack Overflow is that the answers generated by ChatGPT are of low quality. To verify this, we conduct a comparative evaluation of human-written and ChatGPT-generated answers. Our methodology employs both automatic comparison and a manual study. Our results suggest that human-written and ChatGPT-generated answers are semantically similar, however, human-written answers outperform ChatGPT-generated ones consistently across multiple aspects, …

Go to article

The Devil Is In The Tails: How Long-Tailed Code Distributions Impact Large Language Models, Xin Zhou, Kisub Kim, Bowen Xu, Jiakun Liu, Donggyun Han, David Lo Sep 2023

The Devil Is In The Tails: How Long-Tailed Code Distributions Impact Large Language Models, Xin Zhou, Kisub Kim, Bowen Xu, Jiakun Liu, Donggyun Han, David Lo

Research Collection School Of Computing and Information Systems

Learning-based techniques, especially advanced Large Language Models (LLMs) for code, have gained considerable popularity in various software engineering (SE) tasks. However, most existing works focus on designing better learning-based models and pay less attention to the properties of datasets. Learning-based models, including popular LLMs for code, heavily rely on data, and the data's properties (e.g., data distribution) could significantly affect their behavior. We conducted an exploratory study on the distribution of SE data and found that such data usually follows a skewed distribution (i.e., long-tailed distribution) where a small number of classes have an extensive collection of samples, while a …

Go to article

Fine-Grained In-Context Permission Classification For Android Apps Using Control-Flow Graph Embedding, Vikas Kumar Malviya, Naing Tun Yan, Chee Wei Leow, Ailys Xynyn Tee, Lwin Khin Shar, Lingxiao Jiang Sep 2023

Fine-Grained In-Context Permission Classification For Android Apps Using Control-Flow Graph Embedding, Vikas Kumar Malviya, Naing Tun Yan, Chee Wei Leow, Ailys Xynyn Tee, Lwin Khin Shar, Lingxiao Jiang

Research Collection School Of Computing and Information Systems

Android is the most popular operating system for mobile devices nowadays. Permissions are a very important part of Android security architecture. Apps frequently need the users’ permission, but many of them only ask for it once—when the user uses the app for the first time—and then they keep and abuse the given permissions. Longing to enhance Android permission security and users’ private data protection is the driving factor behind our approach to explore fine-grained contextsensitive permission usage analysis and thereby identify misuses in Android apps. In this work, we propose an approach for classifying the fine-grained permission uses for each …

Go to article

Arduinoprog: Towards Automating Arduino Programming, Imam Nur Bani Yusuf, Diyanah Binte Abdul Jamal, Lingxiao Jiang Sep 2023

Arduinoprog: Towards Automating Arduino Programming, Imam Nur Bani Yusuf, Diyanah Binte Abdul Jamal, Lingxiao Jiang

Research Collection School Of Computing and Information Systems

Writing code for Arduino poses unique challenges. A developer 1) needs hardware-specific knowledge about the interface configuration between the Arduino controller and the I/Ohardware, 2) identifies a suitable driver library for the I/O hardware, and 3) follows certain usage patterns of the driver library in order to use them properly. In this work, based on a study of real-world user queries posted in the Arduino forum, we propose ArduinoProg to address such challenges. ArduinoProg consists of three components, i.e., Library Retriever, Configuration Classifier, and Pattern Generator. Given a query, Library Retriever retrieves library names relevant to the I/O hardware identified …

Go to article

Are We Ready To Embrace Generative Ai For Software Q&A?, Bowen Xu, Thanh-Dat Nguyen, Thanh Le-Cong, Thong Hoang, Jiakun Liu, Kisub Kim, Chen Gong, Changan Niu, Chenyu Wang, David Lo, David Lo Sep 2023

Are We Ready To Embrace Generative Ai For Software Q&A?, Bowen Xu, Thanh-Dat Nguyen, Thanh Le-Cong, Thong Hoang, Jiakun Liu, Kisub Kim, Chen Gong, Changan Niu, Chenyu Wang, David Lo, David Lo

Research Collection School Of Computing and Information Systems

Stack Overflow, the world's largest software Q&A (SQA) website, is facing a significant traffic drop due to the emergence of generative AI techniques. ChatGPT is banned by Stack Overflow after only 6 days from its release. The main reason provided by the official Stack Overflow is that the answers generated by ChatGPT are of low quality. To verify this, we conduct a comparative evaluation of human-written and ChatGPT-generated answers. Our methodology employs both automatic comparison and a manual study. Our results suggest that human-written and ChatGPT-generated answers are semantically similar, however, human-written answers outperform ChatGPT-generated ones consistently across multiple aspects, …

Go to article

Sparsity Brings Vulnerabilities: Exploring New Metrics In Backdoor Attacks, Jianwen Tian, Kefan Qiu, Debin Gao, Zhi Wang, Xiaohui Kuang, Gang Zhao Aug 2023

Sparsity Brings Vulnerabilities: Exploring New Metrics In Backdoor Attacks, Jianwen Tian, Kefan Qiu, Debin Gao, Zhi Wang, Xiaohui Kuang, Gang Zhao

Research Collection School Of Computing and Information Systems

Nowadays, using AI-based detectors to keep pace with the fast iterating of malware has attracted a great attention. However, most AI-based malware detectors use features with vast sparse subspaces to characterize applications, which brings significant vulnerabilities to the model. To exploit this sparsityrelated vulnerability, we propose a clean-label backdoor attack consisting of a dissimilarity metric-based candidate selection and a variation ratio-based trigger construction. The proposed backdoor is verified on different datasets, including a Windows PE dataset, an Android dataset with numerical and boolean feature values, and a PDF dataset. The experimental results show that the attack can slash the accuracy …

Go to article

The Future Of Cryptocurrency And Blockchain Technology In Finance, Wanyi Wong, Alan @ Ali Madjelisi Megargel Aug 2023

The Future Of Cryptocurrency And Blockchain Technology In Finance, Wanyi Wong, Alan @ Ali Madjelisi Megargel

Research Collection School Of Computing and Information Systems

Cryptocurrencies have been all the rage in recent years, with many being drawn to their appeal as speculative investment assets. Its proponents also champion the secure and decentralised nature of the technology it is based on, called the blockchain. Given the secure nature of blockchain technology, the idea of adopting cryptocurrencies as legal tender currency has also been mooted and experimented with – with the most famous example being the Central American nation of El Salvador’s bold move to adopting the cryptocurrency Bitcoin as legal tender in September 2021. In theory, this would provide a solution to the high transaction …

Go to article

Nodemedic: End-To-End Analysis Of Node.Js Vulnerabilities With Provenance Graphs, Darion Cassel, Wai Tuck Wong, Limin Jia Jul 2023

Nodemedic: End-To-End Analysis Of Node.Js Vulnerabilities With Provenance Graphs, Darion Cassel, Wai Tuck Wong, Limin Jia

Research Collection School Of Computing and Information Systems

Packages in the Node.js ecosystem often suffer from serious vulnerabilities such as arbitrary command injection and code execution. Existing taint analysis tools fall short in providing an end-to-end infrastructure for automatically detecting and triaging these vulnerabilities.We develop NodeMedic, an end-to-end analysis infrastructure that automates test driver creation, performs precise yet scalable dynamic taint propagation via algorithmically tuned propagation policies, and exposes taint provenance information as a provenance graph. Using provenance graphs we develop two post-detection analyses: automated constraint-based exploit synthesis to confirm vulnerabilities; Attack-defense-tree-based rating of flow exploitability.We demonstrate the effectiveness of NodeMedic through a large-scale evaluation of 10,000 Node.js …

Go to article

Context-Aware Neural Fault Localization, Zhuo Zhang, Xiaoguang Mao, Meng Yan, Xin Xia, David Lo, David Lo Jul 2023

Context-Aware Neural Fault Localization, Zhuo Zhang, Xiaoguang Mao, Meng Yan, Xin Xia, David Lo, David Lo

Research Collection School Of Computing and Information Systems

Numerous fault localization techniques identify suspicious statements potentially responsible for program failures by discovering the statistical correlation between test results (i.e., failing or passing) and the executions of the different statements of a program (i.e., covered or not covered). They rarely incorporate a failure context into their suspiciousness evaluation despite the fact that a failure context showing how a failure is produced is useful for analyzing and locating faults. Since a failure context usually contains the transitive relationships among the statements of causing a failure, its relationship complexity becomes one major obstacle for the context incorporation in suspiciousness evaluation of …

Go to article

Chatgpt, Can You Generate Solutions For My Coding Exercises? An Evaluation On Its Effectiveness In An Undergraduate Java Programming Course, Eng Lieh Ouh, Benjamin Gan, Kyong Jin Shim, Swavek Wlodkowski Jul 2023

Chatgpt, Can You Generate Solutions For My Coding Exercises? An Evaluation On Its Effectiveness In An Undergraduate Java Programming Course, Eng Lieh Ouh, Benjamin Gan, Kyong Jin Shim, Swavek Wlodkowski

Research Collection School Of Computing and Information Systems

In this study, we assess the efficacy of employing the ChatGPT language model to generate solutions for coding exercises within an undergraduate Java programming course. ChatGPT, a large-scale, deep learning-driven natural language processing model, is capable of producing programming code based on textual input. Our evaluation involves analyzing ChatGPT-generated solutions for 80 diverse programming exercises and comparing them to the correct solutions. Our findings indicate that ChatGPT accurately generates Java programming solutions, which are characterized by high readability and well-structured organization. Additionally, the model can produce alternative, memory-efficient solutions. However, as a natural language processing model, ChatGPT struggles with coding …

Go to article

Semantic-Based Neural Network Repair, Richard Schumi, Jun Sun Jul 2023

Semantic-Based Neural Network Repair, Richard Schumi, Jun Sun

Research Collection School Of Computing and Information Systems

Recently, neural networks have spread into numerous fields including many safety-critical systems. Neural networks are built (and trained) by programming in frameworks such as TensorFlow and PyTorch. Developers apply a rich set of pre-defined layers to manually program neural networks or to automatically generate them (e.g., through AutoML). Composing neural networks with different layers is error-prone due to the non-trivial constraints that must be satisfied in order to use those layers. In this work, we propose an approach to automatically repair erroneous neural networks. The challenge is in identifying a minimal modification to the network so that it becomes valid. …

Go to article

Seed Selection For Testing Deep Neural Networks, Yuhan Zhi, Xiaofei Xie, Chao Shen, Jun Sun, Xiaoyu Zhang, Xiaohong Guan Jul 2023

Seed Selection For Testing Deep Neural Networks, Yuhan Zhi, Xiaofei Xie, Chao Shen, Jun Sun, Xiaoyu Zhang, Xiaohong Guan

Research Collection School Of Computing and Information Systems

Deep learning (DL) has been applied in many applications. Meanwhile, the quality of DL systems is becoming a big concern. To evaluate the quality of DL systems, a number of DL testing techniques have been proposed. To generate test cases, a set of initial seed inputs are required. Existing testing techniques usually construct seed corpus by randomly selecting inputs from training or test dataset. Till now, there is no study on how initial seed inputs affect the performance of DL testing and how to construct an optimal one. To fill this gap, we conduct the first systematic study to evaluate …

Go to article

Silent Compiler Bug De-Duplication Via Three-Dimensional Analysis, Chen Yang, Junjie Chen, Xingyu Fan, Jiajun Jiang, Jun Sun Jul 2023

Silent Compiler Bug De-Duplication Via Three-Dimensional Analysis, Chen Yang, Junjie Chen, Xingyu Fan, Jiajun Jiang, Jun Sun

Research Collection School Of Computing and Information Systems

Compiler testing is an important task for assuring the quality of compilers, but investigating test failures is very time-consuming. This is because many test failures are caused by the same compiler bug (known as bug duplication problem). In particular, this problem becomes much more challenging on silent compiler bugs (also called wrong code bugs), since these bugs can provide little information (unlike crash bugs that can produce error messages) for bug de-duplication. In this work, we propose a novel technique (called D3) to solve the duplication problem on silent compiler bugs. Its key insight is to characterize the silent bugs …

Go to article

Beyond "Protected" And "Private": An Empirical Security Analysis Of Custom Function Modifiers In Smart Contracts, Yuzhou Fang, Daoyuan Wu, Xiao Yi, Shuai Wang, Yufan Chen, Mengjie Chen, Yang Liu, Lingxiao Jiang Jul 2023

Beyond "Protected" And "Private": An Empirical Security Analysis Of Custom Function Modifiers In Smart Contracts, Yuzhou Fang, Daoyuan Wu, Xiao Yi, Shuai Wang, Yufan Chen, Mengjie Chen, Yang Liu, Lingxiao Jiang

Research Collection School Of Computing and Information Systems

A smart contract is a piece of application-layer code running on blockchain ledgers and it provides programmatic logic via transaction-based execution of pre-defined functions. Smart contract functions are by default invokable by any party. To safeguard them, the mainstream smart contract language, i.e., Solidity of the popular Ethereum blockchain, proposed a unique language-level keyword called “modifier,” which allows developers to define custom function access control policies beyond the traditional “protected” and “private” modifiers in classic programming languages.In this paper, we aim to conduct a large-scale security analysis of the modifiers used in real-world Ethereum smart contracts. To achieve this, we …

Go to article

Synthesizing Speech Test Cases With Text-To-Speech? An Empirical Study On The False Alarms In Automated Speech Recognition Testing, Julia Kaiwen Lau, Kelvin Kai Wen Kong, Julian Hao Yong, Per Hoong Tan, Zhou Yang, Zi Qian Yong, Joshua Chern Wey Low, Chun Yong Chong, Mei Kuan Lim, David Lo Jul 2023

Synthesizing Speech Test Cases With Text-To-Speech? An Empirical Study On The False Alarms In Automated Speech Recognition Testing, Julia Kaiwen Lau, Kelvin Kai Wen Kong, Julian Hao Yong, Per Hoong Tan, Zhou Yang, Zi Qian Yong, Joshua Chern Wey Low, Chun Yong Chong, Mei Kuan Lim, David Lo

Research Collection School Of Computing and Information Systems

Recent studies have proposed the use of Text-To-Speech (TTS) systems to automatically synthesise speech test cases on a scale and uncover a large number of failures in ASR systems. However, the failures uncovered by synthetic test cases may not reflect the actual performance of an ASR system when it transcribes human audio, which we refer to as false alarms. Given a failed test case synthesised from TTS systems, which consists of TTS-generated audio and the corresponding ground truth text, we feed the human audio stating the same text to an ASR system. If human audio can be correctly transcribed, an …

Go to article

First Prev Next Last