Open Access. Powered by Scholars. Published by Universities.®
Physical Sciences and Mathematics Commons™
Open Access. Powered by Scholars. Published by Universities.®
Articles 1 - 17 of 17
Full-Text Articles in Physical Sciences and Mathematics
Robust Test Selection For Deep Neural Networks, Weifeng Sun, Meng Yan, Zhongxin Liu, David Lo
Robust Test Selection For Deep Neural Networks, Weifeng Sun, Meng Yan, Zhongxin Liu, David Lo
Research Collection School Of Computing and Information Systems
Deep Neural Networks (DNNs) have been widely used in various domains, such as computer vision and software engineering. Although many DNNs have been deployed to assist various tasks in the real world, similar to traditional software, they also suffer from defects that may lead to severe outcomes. DNN testing is one of the most widely used methods to ensure the quality of DNNs. Such method needs rich test inputs with oracle information (expected output) to reveal the incorrect behaviors of a DNN model. However, manually labeling all the collected test inputs is a labor-intensive task, which delays the quality assurance …
Constructing Cyber-Physical System Testing Suites Using Active Sensor Fuzzing, Fan. Zhang, Qianmei. Wu, Bohan. Xuan, Yuqi. Chen, Wei. Lin, Christopher M. Poskitt, Jun Sun, Binbin. Chen
Constructing Cyber-Physical System Testing Suites Using Active Sensor Fuzzing, Fan. Zhang, Qianmei. Wu, Bohan. Xuan, Yuqi. Chen, Wei. Lin, Christopher M. Poskitt, Jun Sun, Binbin. Chen
Research Collection School Of Computing and Information Systems
Cyber-physical systems (CPSs) automating critical public infrastructure face a pervasive threat of attack, motivating research into different types of countermeasures. Assessing the effectiveness of these countermeasures is challenging, however, as benchmarks are difficult to construct manually, existing automated testing solutions often make unrealistic assumptions, and blindly fuzzing is ineffective at finding attacks due to the enormous search spaces and resource requirements. In this work, we propose active sensor fuzzing , a fully automated approach for building test suites without requiring any a prior knowledge about a CPS. Our approach employs active learning techniques. Applied to a real-world water treatment system, …
Generative Model-Based Testing On Decision-Making Policies, Zhuo Li, Xiongfei Wu, Derui Zhu, Mingfei Cheng, Siyuan Chen, Fuyuan Zhang, Xiaofei Xie, Lei Ma, Jianjun Zhao
Generative Model-Based Testing On Decision-Making Policies, Zhuo Li, Xiongfei Wu, Derui Zhu, Mingfei Cheng, Siyuan Chen, Fuyuan Zhang, Xiaofei Xie, Lei Ma, Jianjun Zhao
Research Collection School Of Computing and Information Systems
The reliability of decision-making policies is urgently important today as they have established the fundamentals of many critical applications, such as autonomous driving and robotics. To ensure reliability, there have been a number of research efforts on testing decision-making policies that solve Markov decision processes (MDPs). However, due to the deep neural network (DNN)-based inherit and infinite state space, developing scalable and effective testing frameworks for decision-making policies still remains open and challenging.In this paper, we present an effective testing framework for decision-making policies. The framework adopts a generative diffusion model-based test case generator that can easily adapt to different …
Specification-Based Autonomous Driving System Testing, Yuan Zhou, Yang Sun, Yun Tang, Yuqi Chen, Jun Sun, Christopher M. Poskitt, Yang Liu, Zijiang Yang
Specification-Based Autonomous Driving System Testing, Yuan Zhou, Yang Sun, Yun Tang, Yuqi Chen, Jun Sun, Christopher M. Poskitt, Yang Liu, Zijiang Yang
Research Collection School Of Computing and Information Systems
Autonomous vehicle (AV) systems must be comprehensively tested and evaluated before they can be deployed. High-fidelity simulators such as CARLA or LGSVL allow this to be done safely in very realistic and highly customizable environments. Existing testing approaches, however, fail to test simulated AVs systematically, as they focus on specific scenarios and oracles (e.g., lane following scenario with the "no collision" requirement) and lack any coverage criteria measures. In this paper, we propose AVUnit, a framework for systematically testing AV systems against customizable correctness specifications. Designed modularly to support different simulators, AVUnit consists of two new languages for specifying dynamic …
Detecting C++ Compiler Front-End Bugs Via Grammar Mutation And Differential Testing, Haoxin Tu, He Jiang, Zhide Zhou, Yixuan Tang, Zhilei Ren, Lei Qiao, Lingxiao Jiang
Detecting C++ Compiler Front-End Bugs Via Grammar Mutation And Differential Testing, Haoxin Tu, He Jiang, Zhide Zhou, Yixuan Tang, Zhilei Ren, Lei Qiao, Lingxiao Jiang
Research Collection School Of Computing and Information Systems
C++ is a widely used programming language and the C++ front-end is a critical part of a C++ compiler. Although many techniques have been proposed to test compilers, few studies are devoted to detecting bugs in C++ compiler. In this study, we take the first step to detect bugs in C++ compiler front-ends. To do so, two main challenges need to be addressed, namely, the acquisition of test programs that are more likely to trigger bugs in compiler front-ends and the bug identification from complicated compiler outputs. In this article, we propose a novel framework named Ccoft to detect bugs …
Demystifying Performance Regressions In String Solvers, Yao Zhang, Xiaofei Xie, Yi Li, Yi Lin, Sen Chen, Yang Liu, Xiaohong Li
Demystifying Performance Regressions In String Solvers, Yao Zhang, Xiaofei Xie, Yi Li, Yi Lin, Sen Chen, Yang Liu, Xiaohong Li
Research Collection School Of Computing and Information Systems
Over the past few years, SMT string solvers have found their applications in an increasing number of domains, such as program analyses in mobile and Web applications, which require the ability to reason about string values. A series of research has been carried out to find quality issues of string solvers in terms of its correctness and performance. Yet, none of them has considered the performance regressions happening across multiple versions of a string solver. To fill this gap, in this paper, we focus on solver performance regressions (SPRs), i.e., unintended slowdowns introduced during the evolution of string solvers. To …
Web Apis: Features, Issues, And Expectations: A Large-Scale Empirical Study Of Web Apis From Two Publicly Accessible Registries Using Stack Overflow And A User Survey, Neng Zhang, Ying Zou, Xin Xia, David Lo, David Lo, Shanping Li
Web Apis: Features, Issues, And Expectations: A Large-Scale Empirical Study Of Web Apis From Two Publicly Accessible Registries Using Stack Overflow And A User Survey, Neng Zhang, Ying Zou, Xin Xia, David Lo, David Lo, Shanping Li
Research Collection School Of Computing and Information Systems
With the increasing adoption of services-oriented computing and cloud computing technologies, web APIs have become the fundamental building blocks for constructing software applications. Web APIs are developed and published on the internet. The functionality of web APIs can be used to facilitate the development of software applications. There are numerous studies on retrieving and recommending candidate web APIs based on user requirements from a large set of web APIs. However, there are very limited studies on the features of web APIs that make them more likely to be used and the issues of using web APIs in practice. Moreover, users' …
Quote: Quality-Oriented Testing For Deep Learning Systems, Jialuo Chen, Jingyi Wang, Xingjun Ma, Youcheng Sun, Jun Sun, Peixin Zhang, Peng Cheng
Quote: Quality-Oriented Testing For Deep Learning Systems, Jialuo Chen, Jingyi Wang, Xingjun Ma, Youcheng Sun, Jun Sun, Peixin Zhang, Peng Cheng
Research Collection School Of Computing and Information Systems
Recently, there has been a significant growth of interest in applying software engineering techniques for the quality assurance of deep learning (DL) systems. One popular direction is deep learning testing, i.e., given a property of test, defects of DL systems are found either by fuzzing or guided search with the help of certain testing metrics. However, recent studies have revealed that the neuron coverage metrics, commonly used by most existing DL testing approaches, are not necessarily correlated with model quality (e.g., robustness, the most studied model property), and are also not an effective measurement on the confidence of the model …
An Empirical Study Of Release Note Production And Usage In Practice, Tingting Bi, Xin Xia, David Lo, John Grundy, Thomas Zimmermann
An Empirical Study Of Release Note Production And Usage In Practice, Tingting Bi, Xin Xia, David Lo, John Grundy, Thomas Zimmermann
Research Collection School Of Computing and Information Systems
The release note is one of the most important software artifacts that serves as a bridge for communication among stakeholders. Release notes contain a set of crucial information, such as descriptions of enhancements, improvements, potential issues, development, evolution, testing, and maintenance of projects throughout the whole development lifestyle. A comprehensive understanding of what makes a good release note and how to write one for different stakeholders would be highly beneficial. However, in practice, the release note is often neglected by stakeholders and has not to date been systematically investigated by researchers. In this paper, we conduct a mixed methods study …
Objsim: Efficient Testing Of Cyber-Physical Systems, Jun Sun, Zijiang Yang
Objsim: Efficient Testing Of Cyber-Physical Systems, Jun Sun, Zijiang Yang
Research Collection School Of Computing and Information Systems
Cyber-physical systems (CPSs) play a critical role in automating public infrastructure and thus attract wide range of attacks. Assessing the effectiveness of defense mechanisms is challenging as realistic sets of attacks to test them against are not always available. In this short paper, we briefly describe smart fuzzing, an automated, machine learning guided technique for systematically producing test suites of CPS network attacks. Our approach uses predictive ma- chine learning models and meta-heuristic search algorithms to guide the fuzzing of actuators so as to drive the CPS into different unsafe physical states. The approach has been proven effective on two …
Recovering Fitness Gradients For Interprocedural Boolean Flags In Search-Based Testing, Yun Lin, Jun Sun, Gordon Fraser, Ziheng Xiu, Ting Liu, Jin Song Dong
Recovering Fitness Gradients For Interprocedural Boolean Flags In Search-Based Testing, Yun Lin, Jun Sun, Gordon Fraser, Ziheng Xiu, Ting Liu, Jin Song Dong
Research Collection School Of Computing and Information Systems
In Search-based Software Testing (SBST), test generation is guided by fitness functions that estimate how close a test case is to reach an uncovered test goal (e.g., branch). A popular fitness function estimates how close conditional statements are to evaluating to true or false, i.e., the branch distance. However, when conditions read Boolean variables (e.g., if(x && y)), the branch distance provides no gradient for the search, since a Boolean can either be true or false. This flag problem can be addressed by transforming individual procedures such that Boolean flags are replaced with numeric comparisons that provide better guidance for …
Learning-Guided Network Fuzzing For Testing Cyber-Physical System Defences, Yuqi Chen, Christopher M. Poskitt, Jun Sun, Sridhar Adepu, Fan Zhang
Learning-Guided Network Fuzzing For Testing Cyber-Physical System Defences, Yuqi Chen, Christopher M. Poskitt, Jun Sun, Sridhar Adepu, Fan Zhang
Research Collection School Of Computing and Information Systems
The threat of attack faced by cyber-physical systems (CPSs), especially when they play a critical role in automating public infrastructure, has motivated research into a wide variety of attack defence mechanisms. Assessing their effectiveness is challenging, however, as realistic sets of attacks to test them against are not always available. In this paper, we propose smart fuzzing, an automated, machine learning guided technique for systematically finding 'test suites' of CPS network attacks, without requiring any knowledge of the system's control programs or physical processes. Our approach uses predictive machine learning models and metaheuristic search algorithms to guide the fuzzing of …
Wuji: Automatic Online Combat Game Testing Using Evolutionary Deep Reinforcement Learning, Yan Zheng, Xiaofei Xie, Ting Su, Lei Ma, Jianye Hao, Zhaopeng Meng, Yang Liu, Ruimin Shen, Yingfeng Chen, Changjie Fan
Wuji: Automatic Online Combat Game Testing Using Evolutionary Deep Reinforcement Learning, Yan Zheng, Xiaofei Xie, Ting Su, Lei Ma, Jianye Hao, Zhaopeng Meng, Yang Liu, Ruimin Shen, Yingfeng Chen, Changjie Fan
Research Collection School Of Computing and Information Systems
—Game testing has been long recognized as a notoriously challenging task, which mainly relies on manual playing and scripting based testing in game industry. Even until recently, automated game testing still remains to be largely untouched niche. A key challenge is that game testing often requires to play the game as a sequential decision process. A bug may only be triggered until completing certain difficult intermediate tasks, which requires a certain level of intelligence. The recent success of deep reinforcement learning (DRL) sheds light on advancing automated game testing, without human competitive intelligent support. However, the existing DRLs mostly focus …
Deepstellar: Model-Based Quantitative Analysis Of Stateful Deep Learning Systems, Xiaoning Du, Xiaofei Xie, Yi Li, Lei Ma, Yang Liu, Jianjun Zhao
Deepstellar: Model-Based Quantitative Analysis Of Stateful Deep Learning Systems, Xiaoning Du, Xiaofei Xie, Yi Li, Lei Ma, Yang Liu, Jianjun Zhao
Research Collection School Of Computing and Information Systems
Deep Learning (DL) has achieved tremendous success in many cutting-edge applications. However, the state-of-the-art DL systems still suffer from quality issues. While some recent progress has been made on the analysis of feed-forward DL systems, little study has been done on the Recurrent Neural Network (RNN)-based stateful DL systems, which are widely used in audio, natural languages and video processing, etc. In this paper, we initiate the very first step towards the quantitative analysis of RNN-based DL systems. We model RNN as an abstract state transition system to characterize its internal behaviors. Based on the abstract model, we design two …
Practical And Effective Sandboxing For Linux Containers, Zhiyuan Wan, David Lo, Xin Xia, Liang Cai
Practical And Effective Sandboxing For Linux Containers, Zhiyuan Wan, David Lo, Xin Xia, Liang Cai
Research Collection School Of Computing and Information Systems
A container is a group of processes isolated from other groups via distinct kernel namespaces and resource allocation quota. Attacks against containers often leverage kernel exploits through the system call interface. In this paper, we present an approach that mines sandboxes and enables fine-grained sandbox enforcement for containers. We first explore the behavior of a container by running test cases and monitor the accessed system calls including types and arguments during testing. We then characterize the types and arguments of system call invocations and translate them into sandbox rules for the container. The mined sandbox restricts the container’s access to …
Delta Debugging Microservice Systems, Xiang Zhou, Xin Peng, Tao Xie, Jun Sun, Wenhai Li, Chao Ji, Dan Ding
Delta Debugging Microservice Systems, Xiang Zhou, Xin Peng, Tao Xie, Jun Sun, Wenhai Li, Chao Ji, Dan Ding
Research Collection School Of Computing and Information Systems
Debugging microservice systems involves the deployment and manipulation of microservice systems on a containerized environment and faces unique challenges due to the high complexity and dynamism of microservices. To address these challenges, in this paper, we propose a debugging approach for microservice systems based on the delta debugging algorithm, which is to minimize failureinducing deltas of circumstances (e.g., deployment, environmental configurations) for effective debugging. Our approach includes novel techniques for defining, deploying/manipulating, and executing deltas following the idea of delta debugging. In particular, to construct a (failing) circumstance space for delta debugging to minimize, our approach defines a set of …
Assertion Generation Through Active Learning, Long H. Pham, Jun Sun, Jun Sun
Assertion Generation Through Active Learning, Long H. Pham, Jun Sun, Jun Sun
Research Collection School Of Computing and Information Systems
Program assertions are useful for many program analysis tasks. They are however often missing in practice. In this work, we develop a novel approach for generating likely assertions automatically based on active learning. Our target is complex Java programs which cannot be symbolically executed (yet). Our key idea is to generate candidate assertions based on test cases and then apply active learning techniques to iteratively improve them. The experiments show that active learning really helps to improve the generated assertions.