Software Engineering Commons^™

An Empirical Study Of Gui Widget Detection For Industrial Mobile Games, Jiaming Ye, Ke Chen, Xiaofei Xie, Lei Ma, Ruochen Huang, Yingfeng Chen, Yinxing Xue, Jianjun Zhao

Research Collection School Of Computing and Information Systems

With the widespread adoption of smartphones in our daily life, mobile games experienced increasing demand over the past years. Meanwhile, the quality of mobile games has been continuously drawing more and more attention, which can greatly affect the player experience. For better quality assurance, general-purpose testing has been extensively studied for mobile apps. However, due to the unique characteristic of mobile games, existing mobile testing techniques may not be directly suitable and applicable. To better understand the challenges in mobile game testing, in this paper, we first initiate an early step to conduct an empirical study towards understanding the challenges …

Ava: Adversarial Vignetting Attack Against Visual Recognition, Binyu Tian, Felix Juefei-Xu, Qing Guo, Xiaofei Xie, Xiaohong Li, Yang Liu

Research Collection School Of Computing and Information Systems

Vignetting is an inherent imaging phenomenon within almost all optical systems, showing as a radial intensity darkening toward the corners of an image. Since it is a common effect for photography and usually appears as a slight intensity variation, people usually regard it as a part of a photo and would not even want to post-process it. Due to this natural advantage, in this work, we study the vignetting from a new viewpoint, i.e., adversarial vignetting attack (AVA), which aims to embed intentionally misleading information into the vignetting and produce a natural adversarial example without noise patterns. This example can …

Biasrv: Uncovering Biased Sentiment Predictions At Runtime, Zhou Yang, Muhammad Hilmi Asyrofi, David Lo

Research Collection School Of Computing and Information Systems

Sentiment analysis (SA) systems, though widely applied in many domains, have been demonstrated to produce biased results. Some research works have been done in automatically generating test cases to reveal unfairness in SA systems, but the community still lacks tools that can monitor and uncover biased predictions at runtime. This paper fills this gap by proposing BiasRV, the first tool to raise an alarm when a deployed SA system makes a biased prediction on a given input text. To implement this feature, BiasRV dynamically extracts a template from an input text and from the template generates gender-discriminatory mutants (semanticallyequivalent texts …

Code Integrity Attestation For Plcs Using Black Box Neural Network Predictions, Yuqi Chen, Christopher M. Poskitt, Jun Sun

Research Collection School Of Computing and Information Systems

Cyber-physical systems (CPSs) are widespread in critical domains, and significant damage can be caused if an attacker is able to modify the code of their programmable logic controllers (PLCs). Unfortunately, traditional techniques for attesting code integrity (i.e. verifying that it has not been modified) rely on firmware access or roots-of-trust, neither of which proprietary or legacy PLCs are likely to provide. In this paper, we propose a practical code integrity checking solution based on privacy-preserving black box models that instead attest the input/output behaviour of PLC programs. Using faithful offline copies of the PLC programs, we identify their most important …

Graph-Based Seed Object Synthesis For Search-Based Unit Testing, Yun Lin, You Seng Ong, Jun Sun, Gordon Fraser, Jin Song Dong

Research Collection School Of Computing and Information Systems

Search-based software testing (SBST) generates tests using search algorithms guided by measurements gauging how far a test case is away from exercising a coverage goal. The effectiveness of SBST largely depends on the continuity and monotonicity of the fitness landscape decided by these measurements and the search operators. Unfortunately, the fitness landscape is challenging when the function under test takes object inputs, as classical measurements hardly provide guidance for constructing legitimate object inputs. To overcome this problem, we propose test seeds, i.e., test code skeletons of legitimate objects which enable the use of classical measurements. Given a target branch in …

Type And Interval Aware Array Constraint Solving For Symbolic Execution, Ziqi Shuai, Zhenbang Chen, Yufeng Zhang, Jun Sun, Ji Wang

Research Collection School Of Computing and Information Systems

Array constraints are prevalent in analyzing a program with symbolic execution. Solving array constraints is challenging due to the complexity of the precise encoding for arrays. In this work, we propose to synergize symbolic execution and array constraint solving. Our method addresses the difficulties in solving array constraints with novel ideas. First, we propose a lightweight method for pre-checking the unsatisfiability of array constraints based on integer linear programming. Second, observing that encoding arrays at the byte-level introduces many redundant axioms that reduce the effectiveness of constraint solving, we propose type and interval aware axiom generation. Note that the type …

Learning-Based Extraction Of First-Order Logic Representations Of Api Directives, Mingwei Liu, Xin Peng, Andrian Marcus, Christoph Treude, Xuefang Bai, Gang Lyu, Jiazhen Xie, Xiaoxin Zhang

Research Collection School Of Computing and Information Systems

Developers often rely on API documentation to learn API directives, i.e., constraints and guidelines related to API usage. Failing to follow API directives may cause defects or improper implementations. Since there are no industry-wide standards on how to document API directives, they take many forms and are often hard to understand by developers or challenging to parse with tools. In this paper, we propose a learning based approach for extracting first-order logic representations of API directives (FOL directives for short). The approach, called LeadFOL, uses a joint learning method to extract atomic formulas by identifying the predicates and arguments involved …

Aware: Eliminating Implicit Bias Using Ar, Shagun Bose, Emma Strauch

Frameless

Implicit Bias is something that happens to real people in real spaces all the time. But we can’t see it. Since, AR allows us to overlay virtual objects in real environments, we tried to leverage AR to make more salient the various ways in which people experience bias.

Knot Theory In Virtual Reality, Donald Lee Price

Masters Theses & Specialist Projects

Throughout the study of Knot Theory, there have been several programmatic solutions to common problems or questions. These solutions have included software to draw knots, software to identify knots, or online databases to look up pre-computed data about knots. We introduce a novel prototype of software used to study knots and links by using Virtual Reality. This software can allow researchers to draw links in 3D, run physics simulations on them, and identify them. This technique has not yet been rigorously explored and we believe it will be of great interest to Knot Theory researchers. The computer code is written …

Material Detection With Thermal Imaging And Computer Vision: Potentials And Limitations, Jared Poe

Graduate Theses and Dissertations

The goal of my masters thesis research is to develop an affordable and mobile infraredbased environmental sensoring system for the control of a servo motor based on material identification. While this sensing could be oriented towards different applications, my thesis is particularly interested in material detection due to the wide range of possible applications in mechanical engineering. Material detection using a thermal mobile camera could be used in manufacturing, recycling or autonomous robotics. For my research, the application that will be focused on is using this material detection to control a servo motor by identifying and sending control inputs based …

On The Generalizability Of Neural Program Models With Respect To Semantic-Preserving Program Transformations, Md Rafiqul Islam Rabin, Nghi D. Q. Bui, Ke Wang, Yijun Yu, Lingxiao Jiang

Research Collection School Of Computing and Information Systems

Context: With the prevalence of publicly available source code repositories to train deep neural network models, neural program models can do well in source code analysis tasks such as predicting method names in given programs that cannot be easily done by traditional program analysis techniques. Although such neural program models have been tested on various existing datasets, the extent to which they generalize to unforeseen source code is largely unknown. Objective: Since it is very challenging to test neural program models on all unforeseen programs, in this paper, we propose to evaluate the generalizability of neural program models with respect …

Deep Transfer Bug Localization, Xuan Huo, Ferdian Thung, Ming Li, David Lo, Shu-Ting Shi

Research Collection School Of Computing and Information Systems

Many projects often receive more bug reports than what they can handle. To help debug and close bug reports, a number of bug localization techniques have been proposed. These techniques analyze a bug report and return a ranked list of potentially buggy source code files. Recent development on bug localization has resulted in the construction of effective supervised approaches that use historical data of manually localized bugs to boost performance. Unfortunately, as highlighted by Zimmermann et al., sufficient bug data is often unavailable for many projects and companies. This raises the need for cross-project bug localization -- the use of …

A Differentially Private Task Planning Framework For Spatial Crowdsourcing, Qian Tao, Yongxin Tong, Shuyuan Li, Yuxiang Zeng, Zimu Zhou, Ke Xu

Research Collection School Of Computing and Information Systems

Spatial crowdsourcing has stimulated various new applications such as taxi calling and food delivery. A key enabler for these spatial crowdsourcing based applications is to plan routes for crowd workers to execute tasks given diverse requirements of workers and the spatial crowdsourcing platform. Despite extensive studies on task planning in spatial crowdsourcing, few have accounted for the location privacy of tasks, which may be misused by an untrustworthy platform. In this paper, we explore efficient task planning for workers while protecting the locations of tasks. Specifically, we define the Privacy-Preserving Task Planning (PPTP) problem, which aims at both total revenue …

Self-Supervised Contrastive Learning For Code Retrieval And Summarization Via Semantic-Preserving Transformations, Duy Quoc Nghi Bui, Yijun Yu, Lingxiao Jiang

Research Collection School Of Computing and Information Systems

We propose Corder, a self-supervised contrastive learning framework for source code model. Corder is designed to alleviate the need of labeled data for code retrieval and code summarization tasks. The pre-trained model of Corder can be used in two ways: (1) it can produce vector representation of code which can be applied to code retrieval tasks that do not have labeled data; (2) it can be used in a fine-tuning process for tasks that might still require label data such as code summarization. The key innovation is that we train the source code model by asking it to recognize similar …

Developing An Effective Targeted Mobile Application To Enhance Transportation Safety And Use Of Active Transportation Modes In Fresno County: The Role Of Application Design & Content, Samer Sarofim

Mineta Transportation Institute

Do pedestrians and cyclists need their own app? Pedestrians and cyclists in Fresno county think so, and this research examined this need and how it relates to the importance of app design. Survey participants (all who regularly use active transportation modes) along with a variety of transportation stakeholders, including the Fresno Council of Government, the California Department of Transportation (Caltrans) District 6, and the City of Fresno — Public Works Department, indicated the importance of designing effective communication tools to enhance the utilization of active transportation modes and to ensure the safety of vulnerable road users. In this study, over …

Rnnrepair: Automatic Rnn Repair Via Model-Based Analysis, Xiaofei Xie, Wenbo Guo, Lei Ma, Wei Le, Jian Wang, Lingjun Zhou, Yang Liu, Xinyu Xing

Research Collection School Of Computing and Information Systems

Deep neural networks are vulnerable to adversarial attacks. Due to their black-box nature, it is rather challenging to interpret and properly repair these incorrect behaviors. This paper focuses on interpreting and repairing the incorrect behaviors of Recurrent Neural Networks (RNNs). We propose a lightweight model-based approach (RNNRepair) to help understand and repair incorrect behaviors of an RNN. Specifically, we build an influence model to characterize the stateful and statistical behaviors of an RNN over all the training data and to perform the influence analysis for the errors. Compared with the existing techniques on influence function, our method can efficiently estimate …

Efficient White-Box Fairness Testing Through Gradient Search, Lingfeng Zhang, Yueling Zhang, Min Zhang

Research Collection School Of Computing and Information Systems

Deep learning (DL) systems are increasingly deployed for autonomous decision-making in a wide range of applications. Apart from the robustness and safety, fairness is also an important property that a well-designed DL system should have. To evaluate and improve individual fairness of a model, systematic test case generation for identifying individual discriminatory instances in the input space is essential. In this paper, we propose a framework EIDIG for efficiently discovering individual fairness violation. Our technique combines a global generation phase for rapidly generating a set of diverse discriminatory seeds with a local generation phase for generating as many individual discriminatory …

Bias Field Poses A Threat To Dnn-Based X-Ray Recognition, Bingyu Tian, Qing Guo, Felix Juefei-Xu, Wen Le Chan, Yupeng Cheng, Xiaohong Li, Xiaofei Xie, Shengchao Qin

Research Collection School Of Computing and Information Systems

Chest X-ray plays a key role in screening and diagnosis of many lung diseases including the COVID-19. Many works construct deep neural networks (DNNs) for chest X-ray images to realize automated and efficient diagnosis of lung diseases. However, bias field caused by the improper medical image acquisition process widely exists in the chest X-ray images while the robustness of DNNs to the bias field is rarely explored, posing a threat to the X-ray-based automated diagnosis system. In this paper, we study this problem based on the adversarial attack and propose a brand new attack, i.e., adversarial bias field attack where …

Stealing Deep Reinforcement Learning Models For Fun And Profit, Kangjie Chen, Shangwei Guo, Tianwei Zhang, Xiaofei Xie, Yang Liu

Research Collection School Of Computing and Information Systems

This paper presents the first model extraction attack against Deep Reinforcement Learning (DRL), which enables an external adversary to precisely recover a black-box DRL model only from its interaction with the environment. Model extraction attacks against supervised Deep Learning models have been widely studied. However, those techniques cannot be applied to the reinforcement learning scenario due to DRL models' high complexity, stochasticity and limited observable information. We propose a novel methodology to overcome the above challenges. The key insight of our approach is that the process of DRL model extraction is equivalent to imitation learning, a well-established solution to learn …

Attack As Defense: Characterizing Adversarial Examples Using Robustness, Zhe Zhao, Guangke Chen, Jingyi Wang, Yiwei Yang, Fu Song, Jun Sun

Research Collection School Of Computing and Information Systems

As a new programming paradigm, deep learning has expanded its application to many real-world problems. At the same time, deep learning based software are found to be vulnerable to adversarial attacks. Though various defense mechanisms have been proposed to improve robustness of deep learning software, many of them are ineffective against adaptive attacks. In this work, we propose a novel characterization to distinguish adversarial examples from benign ones based on the observation that adversarial examples are significantly less robust than benign ones. As existing robustness measurement does not scale to large networks, we propose a novel defense framework, named attack …

Towards A Large-Scale Intelligent Mobile-Argumentation And Discovering Arguments, Controversial Topics And Topic-Oriented Focal Sets In Cyber-Argumentation, Najla Althuniyan

Graduate Theses and Dissertations

User-generated content (UGC) platforms host different forms of information, such as audio, video, pictures, and text. They have many online applications, such as social media, blogs, photo and video sharing, customer reviews, debate, and deliberation platforms. Usually, the content of these platforms is provided and consumed by users. Most of these platforms, mainly social media and blogs, are often used for online discussion. These platforms offer tools for users to share and express opinions. Commonly, people from different backgrounds and origins discuss opinions about various issues over the Internet. Furthermore, discussions among users contain substantial information from which knowledge about …

Design And Development Of Techniques To Ensure Integrity In Fog Computing Based Databases, Abdulwahab Fahad S. Alazeb

Graduate Theses and Dissertations

The advancement of information technology in coming years will bring significant changes to the way sensitive data is processed. But the volume of generated data is rapidly growing worldwide. Technologies such as cloud computing, fog computing, and the Internet of things (IoT) will offer business service providers and consumers opportunities to obtain effective and efficient services as well as enhance their experiences and services; increased availability and higher-quality services via real-time data processing augment the potential for technology to add value to everyday experiences. This improves human life quality and easiness. As promising as these technological innovations, they are prone …

Automated Privacy Protection For Mobile Device Users And Bystanders In Public Spaces, David Darling

Graduate Theses and Dissertations

As smartphones have gained popularity over recent years, they have provided usersconvenient access to services and integrated sensors that were previously only available through larger, stationary computing devices. This trend of ubiquitous, mobile devices provides unparalleled convenience and productivity for users who wish to perform everyday actions such as taking photos, participating in social media, reading emails, or checking online banking transactions. However, the increasing use of mobile devices in public spaces by users has negative implications for their own privacy and, in some cases, that of bystanders around them.

Specifically, digital photography trends in public have negative implications for …

Emotioncues: Emotion-Oriented Visual Summarization Of Classroom Videos, Haipeng Zeng, Xinhuan Shu, Yanbang Wang, Yong Wang, Liguo Zhang, Ting-Chuen Pong, Huamin Qu

Research Collection School Of Computing and Information Systems

Analyzing students' emotions from classroom videos can help both teachers and parents quickly know the engagement of students in class. The availability of high-definition cameras creates opportunities to record class scenes. However, watching videos is time-consuming, and it is challenging to gain a quick overview of the emotion distribution and find abnormal emotions. In this paper, we propose EmotionCues, a visual analytics system to easily analyze classroom videos from the perspective of emotion summary and detailed analysis, which integrates emotion recognition algorithms with visualizations. It consists of three coordinated views: a summary view depicting the overall emotions and their dynamic …

Vibransee: Enabling Simultaneous Visible Light Communication And Sensing, Ila Nitin Gokarn, Archan Misra

Research Collection School Of Computing and Information Systems

Driven by the ubiquitous proliferation of low-cost LED luminaires, visible light communication (VLC) has been established as a high-speed communications technology based on the high-frequency modulation of an optical source. In parallel, Visible Light Sensing (VLS) has recently demonstrated how vision-based at-a-distance sensing of mechanical vibrations (e.g., of factory equipment) can be performed using high frequency optical strobing. However, to date, exemplars of VLC and VLS have been explored in isolation, without consideration of their mutual dependencies. In this work, we explore whether and how high-throughput VLC and high-coverage VLS can be simultaneously supported. We first demonstrate the existence of …

Windows Kernel Hijacking Is Not An Option: Memoryranger Comes To The Rescue Again, Igor Korkin

Journal of Digital Forensics, Security and Law

The security of a computer system depends on OS kernel protection. It is crucial to reveal and inspect new attacks on kernel data, as these are used by hackers. The purpose of this paper is to continue research into attacks on dynamically allocated data in the Windows OS kernel and demonstrate the capacity of MemoryRanger to prevent these attacks. This paper discusses three new hijacking attacks on kernel data, which are based on bypassing OS security mechanisms. The first two hijacking attacks result in illegal access to files open in exclusive access. The third attack escalates process privileges, without applying …

A Configurable Social Network For Running Irb-Approved Experiments, Mihovil Mandic

Dartmouth College Undergraduate Theses

Our world has never been more connected, and the size of the social media landscape draws a great deal of attention from academia. However, social networks are also a growing challenge for the Institutional Review Boards concerned with the subjects’ privacy. These networks contain a monumental variety of personal information of almost 4 billion people, allow for precise social profiling, and serve as a primary news source for many users. They are perfect environments for influence operations that are becoming difficult to defend against. Motivated to study online social influence via IRB-approved experiments, we designed and implemented a flexible, scalable, …

Promoting And Teaching Responsible Leadership In Software Engineering, Devender Goyal, Luiz Fernando Capretz

Electrical and Computer Engineering Publications

As software and computer technology is becoming more prominent and pervasive in all spheres of life, many researchers and industry folks are realizing the importance of teaching soft skills and values to CS and SE students. Many researchers and leaders, from both academic and non-academic world, are also calling for software researchers and practitioners to seriously consider human values, like respect, integrity, compassion, justice, and honesty when building software, both for greater social good and also for financial considerations. In this paper, we propose and wish to promote teaching soft skills, values, and responsibilities to students, which we term as …

M^3, Catherine Mingmin Wei, Syrsha Anne Harvey

Computer Science and Software Engineering

M^3 aims to raise students' interest in physics through an interactive and fun video game. Originally planned as a segment in the Cal Poly SLO Seeds in STEM workshop, M^3 is designed with middle and high school students as the intended audience.

Riding Through The Silver Tsunami: A Data Driven Approach To Improve Senior Citizens’ Engagement With Community Senior Activity Centres, Joshua Jie Feng Lam, Hwee-Pink Tan

Research Collection School Of Computing and Information Systems

In Singapore, 1 in 4 persons will be elderly by 2030 In preparation for the Silver Tsunami, the Singapore government and community care providers have collaborations to promote active, independent living amongst elders Current implementation of data driven population health is focused on well being indices using data collected from the general population There is no literature on the use of data analytics in assessing elder