Software Engineering Commons^™

Do Comments And Expertise Still Matter? An Experiment On Programmers’ Adoption Of Ai-Generated Javascript Code, Changwen Li, Christoph Treude, Ofir Turel

Research Collection School Of Computing and Information Systems

This paper investigates the factors influencing programmers’ adoption of AI-generated JavaScript code recommendations within the context of lightweight, function-level programming tasks. It extends prior research by (1) utilizing objective (as opposed to the typically self-reported) measurements for programmers’ adoption of AI-generated code and (2) examining whether AI-generated comments added to code recommendations and development expertise drive AI-generated code adoption. We tested these potential drivers in an online experiment with 173 programmers. Participants were asked to answer some questions to demonstrate their level of development expertise. Then, they were asked to solve a LeetCode problem without AI support. After attempting to …

Interoperability In Deep Learning: A User Survey And Failure Analysis Of Onnx Model Converters, Purvish Jajal, Wenxin Jiang, Arav Tewari, Erik Kocinare, Joseph Woo, Anusha Sarraf, Yung-Hsiang Lu, George Thiruvathukal, James C. Davis

Computer Science: Faculty Publications and Other Works

Software engineers develop, fine-tune, and deploy deep learning (DL) models using a variety of development frameworks and runtime environments. DL model converters move models between frameworks and to runtime environments. Conversion errors compromise model quality and disrupt deployment. However, the failure characteristics of DL model converters are unknown, adding risk when using DL interoperability technologies. This paper analyzes failures in DL model converters. We survey software engineers about DL interoperability tools, use cases, and pain points (N=92). Then, we characterize failures in model converters associated with the main interoperability tool, ONNX (N=200 issues in PyTorch and TensorFlow). Finally, we formulate …

Challenges And Practices Of Deep Learning Model Reengineering: A Case Study On Computer Vision, Wenxin Jiang, Vishnu Banna, Naveen Vivek, Abhinav Goel, Nicholas Synovic, George K. Thiruvathukal, James C. Davis

Computer Science: Faculty Publications and Other Works

Many engineering organizations are reimplementing and extending deep neural networks from the research community. We describe this process as deep learning model reengineering. Deep learning model reengineering — reusing, replicating, adapting, and enhancing state-of-the-art deep learning approaches — is challenging for reasons including under-documented reference models, changing requirements, and the cost of implementation and testing.

Revisiting The Identification Of The Co-Evolution Of Production And Test Code, Weifeng Sun, Meng Yan, Zhongxin Liu, Xin Xia, Yan Lei, David Lo

Research Collection School Of Computing and Information Systems

Many software processes advocate that the test code should co-evolve with the production code. Prior work usually studies such co-evolution based on production-test co-evolution samples mined from software repositories. A production-test co-evolution sample refers to a pair of a test code change and a production code change where the test code change triggers or is triggered by the production code change. The quality of the mined samples is critical to the reliability of research conclusions. Existing studies mined production-test co-evolution samples based on the following assumption: if a test class and its associated production class change together in one commit, …

Ptmtorrent: A Dataset For Mining Open-Source Pre-Trained Model Packages, Wenxin Jiang, Nicholas Synovic, Purvish Jajal, Taylor R. Schorlemmer, Arav Tewari, Bhavesh Pareek, George K. Thiruvathukal, James C. Davis

Computer Science: Faculty Publications and Other Works

Due to the cost of developing and training deep learning models from scratch, machine learning engineers have begun to reuse pre-trained models (PTMs) and fine-tune them for downstream tasks. PTM registries known as “model hubs” support engineers in distributing and reusing deep learning models. PTM packages include pre-trained weights, documentation, model architectures, datasets, and metadata. Mining the information in PTM packages will enable the discovery of engineering phenomena and tools to support software engineers. However, accessing this information is difficult — there are many PTM registries, and both the registries and the individual packages may have rate limiting for accessing …

An Empirical Study Of Pre-Trained Model Reuse In The Hugging Face Deep Learning Model Registry, Wenxin Jiang, Nicholas Synovic, Matt Hyatt, Taylor R. Schorlemmer, Rohan Sethi, Yung-Hsiang Lu, George K. Thiruvathukal, James C. Davis

Department of Electrical and Computer Engineering Faculty Publications

Deep Neural Networks (DNNs) are being adopted as components in software systems. Creating and specializing DNNs from scratch has grown increasingly difficult as state-of-the-art architectures grow more complex. Following the path of traditional software engineering, machine learning engineers have begun to reuse large-scale pre-trained models (PTMs) and fine-tune these models for downstream tasks. Prior works have studied reuse practices for traditional software packages to guide software engineers towards better package maintenance and dependency management. We lack a similar foundation of knowledge to guide behaviors in pre-trained model ecosystems.

In this work, we present the first empirical investigation of PTM reuse. …

A Tool-Supported Metamodel For Program Bugfix Analysis In Empirical Software Engineering, Manal Zneit

Theses and Dissertations

This thesis describes a software modeling approach aimed at addressing empirical studies in software engineering. We build a metamodel that provides an overview of the taxonomy of program bugfixes in deep learning programs. For modeling purposes, we present a prototype tool that is an implementation of the model-driven techniques presented.

Reflections On Software Failure Analysis, Paschal C. Amusuo, Aishwarya Sharma, Siddharth R. Rao, Abbey Vincent, James C. Davis

Department of Electrical and Computer Engineering Faculty Publications

Failure studies are important in revealing the root causes, behaviors, and life cycle of defects in software systems. These studies either focus on understanding the characteristics of defects in specific classes of systems or the characteristics of a specific type of defect in the systems it manifests in. Failure studies have influenced various software engineering research directions, especially in the area of software evolution, defect detection, and program repair.

In this paper, we reflect on the conduct of failure studies in software engineering. We reviewed a sample of 52 failure study papers. We identified several recurring problems in these studies, …

Exploiting Input Sanitization For Regex Denial Of Service, Efe Barlas, Xin Du, James C. Davis

Department of Electrical and Computer Engineering Faculty Publications

Web services use server-side input sanitization to guard against harmful input. Some web services publish their sanitization logic to make their client interface more usable, e.g., allowing clients to debug invalid requests locally. However, this usability practice poses a security risk. Specifically, services may share the regexes they use to sanitize input strings — and regex-based denial of service (ReDoS) is an emerging threat. Although prominent service outages caused by ReDoS have spurred interest in this topic, we know little about the degree to which live web services are vulnerable to ReDoS.

In this paper, we conduct the first black-box …

Discrepancies Among Pre-Trained Deep Neural Networks: A New Threat To Model Zoo Reliability, Diego Montes, Pongpatapee Peerapatanapokin, Jeff Schultz, Chengjun Guo, Wenxin Jiang, James C. Davis

Department of Electrical and Computer Engineering Faculty Publications

Training deep neural networks (DNNs) takes significant time and resources. A practice for expedited deployment is to use pre-trained deep neural networks (PTNNs), often from model zoos.collections of PTNNs; yet, the reliability of model zoos remains unexamined. In the absence of an industry standard for the implementation and performance of PTNNs, engineers cannot confidently incorporate them into production systems. As a first step, discovering potential discrepancies between PTNNs across model zoos would reveal a threat to model zoo reliability. Prior works indicated existing variances in deep learning systems in terms of accuracy. However, broader measures of reliability for PTNNs from …

An Empirical Study On The Impact Of Deep Parameters On Mobile App Energy Usage, Qiang Xu, James C. Davis, Y Charlie Hu, Abhilash Jindal

Department of Electrical and Computer Engineering Faculty Publications

Improving software performance through configuration parameter tuning is a common activity during software maintenance. Beyond traditional performance metrics like latency, mobile app developers are interested in reducing app energy usage. Some mobile apps have centralized locations for parameter tuning, similar to databases and operating systems, but it is common for mobile apps to have hundreds of parameters scattered around the source code. The correlation between these "deep" parameters and app energy usage is unclear. Researchers have studied the energy effects of deep parameters in specific modules, but we lack a systematic understanding of the energy impact of mobile deep parameters. …

An Empirical Study Of Developers' Discussions About Security Challenges Of Different Programming Languages, Roland Croft, Yongzheng Xie, Mansooreh Zahedi, Muhammad Ali Babar, Christoph Treude

Research Collection School Of Computing and Information Systems

In collaborative software development projects, work items are used as a mechanism to coordinate tasks and track shared development work. In this paper, we explore how “tagging,” a lightweight social computing mechanism, is used to communicate matters of concern in the management of development tasks. We present the results from two empirical studies over 36 and 12 months, respectively, on how tagging has been adopted and what role it plays in the development processes of several professional development projects with more than 1,000 developers in total. Our research shows that the tagging mechanism was eagerly adopted by the teams, and …

Comparing The Popularity Of Testing Careers Among Canadian, Indian, Chinese, And Malaysian Students, Luiz Fernando Capretz, Pradeep Waychal, Jingdong Jia, Shuib Basri

Electrical and Computer Engineering Publications

This study attempts to understand motivators and de-motivators that influence the decisions of software students to take up and sustain software testing careers across four different countries, Canada, India, China, and Malaysia. Towards that end, we have developed a cross-sectional, but simple, survey-based instrument. In this study we investigated how software engineering and computer science students perceive and value what they do and their environmental settings. This study found that very few students are keen to take up software testing careers - why is this happening with such an important task in the software life cycle? The common advantages of …

Characterization And Prediction Of Questions Without Accepted Answers On Stack Overflow, Mohamad Yazdaninia, David Lo, Ashkan Sami

Research Collection School Of Computing and Information Systems

A fast and effective approach to obtain information regarding software development problems is to search them to find similar solved problems or post questions on community question answering (CQA) websites. Solving coding problems in a short time is important, so these CQAs have a considerable impact on the software development process. However, if developers do not get their expected answers, the websites will not be useful, and software development time will increase. Stack Overflow is the most popular CQA concerning programming problems. According to its rules, the only sign that shows a question poser has achieved the desired answer is …

Selecting Third-Party Libraries: The Practitioners' Perspective, Enrique Larios Vargas, Maurício Aniche, Christoph Treude, Magiel Bruntink, Georgios Gousios

Research Collection School Of Computing and Information Systems

The selection of third-party libraries is an essential element of virtually any software development project. However, deciding which libraries to choose is a challenging practical problem. Selecting the wrong library can severely impact a software project in terms of cost, time, and development effort, with the severity of the impact depending on the role of the library in the software architecture, among others. Despite the importance of following a careful library selection process, in practice, the selection of third-party libraries is still conducted in an ad-hoc manner, where dozens of factors play an influential role in the decision. In this …

Comparing The Popularity Of Testing Career Among Canadian, Chinese, And Indian Students, Luiz Fernando Capretz, Pradeep Waychal, Jingdong Jia

Electrical and Computer Engineering Publications

Despite its importance, software testing is, arguably, the least understood part of the software life cycle and still the toughest to perform correctly. Many researchers and practitioners have been working to address the situation. However, most of the studies focus on the process and technology dimensions and only a few on the human dimension of testing, in spite of the reported relevance of human aspects of software testing. Testers need to understand various stakeholders’ explicit and implicit requirements, be aware of how developers work individually and in teams, and develop skills to report test results wisely to stakeholders. These multifaceted …

Studies On The Software Testing Profession, Luiz Fernando Capretz, Pradeep Waychal, Jingdong Jia, Daniel Varona, Yadira Tejeda Saldaña

Electrical and Computer Engineering Publications

This paper attempts to understand motivators and de-motivators that influence the decisions of software professionals to take up and sustain software testing careers across four different countries, i.e. Canada, China, Cuba, and India. The research question can be framed as “How many software professionals across different geographies are keen to take up testing careers, and what are the reasons for their choices?” Towards that, we developed a cross-sectional but simple survey-based instrument. In this study we investigated how software testers perceived and valued what they do and their environmental settings. The study pointed out the importance of visualizing software testing …

An Empirical Study Of Css Code Smells In Web Frameworks, Tobias Paul Bleisch

Master's Theses

Cascading Style Sheets (CSS) has become essential to front-end web development for the specification of style. But despite its simple syntax and the theoretical advantages attained through the separation of style from content and behavior, CSS authoring today is regarded as a complex task. As a result, developers are increasingly turning to CSS preprocessor languages and web frameworks to aid in development. However, previous studies show that even highly popular websites which are known to be developed with web frameworks contain CSS code smells such as duplicated rules and hard-coded values. Such code smells have the potential to cause adverse …

Introducing Programmers To Pair Programming: A Controlled Experiment, A. S. M. Sajeev, Subhajit Datta

Research Collection School Of Computing and Information Systems

Pair programming is a key characteristic of the Extreme Programming (XP) method. Through a controlled experiment we investigate pair programming behaviour of programmers without prior experience in XP. The factors investigated are: (a) characteristics of pair programming that are less favored (b) perceptions of team effectiveness and how they relate to product quality, and (c) whether it is better to train a pair by giving routine tasks first or by giving complex tasks first. Our results show that: (a) the least liked aspects of pair programming were having to share the screen, keyboard and mouse, and having to switch between …

Users’ Perception Of Open Source Usability: An Empirical Study, Arif Raza, Luiz Fernando Capretz, Faheem Ahmed

Electrical and Computer Engineering Publications

The number of open source software (OSS) users has increased in recent years. No longer are they limited to technically adept software developers. Many believe that the OSS market share could increase tremendously provided OSS had systems that were easier to use. Although examples of good usable open source software exist, it is agreed that OSS can be made more usable. This study presents an empirical investigation to study the impact of some key factors on OSS usability from the end users’ point of view. The research model studies and establishes the relationship between the key usability factors from the …

An Empirical Study Of Open Source Software Usability – The Industrial Perspective, Arif Raza, Luiz Fernando Capretz, Faheem Ahmed

Electrical and Computer Engineering Publications

Recent years have seen a sharp increase in the use of open source projects by common novice users; Open Source Software (OSS) is thus no longer a reserved arena for software developers and computer gurus. Although user-centered designs are gaining popularity in OSS, usability is still not considered one of the prime objectives in many design scenarios. This paper analyzes industry users’ perception of usability factors, including understandability, learnability, operability, and attractiveness on OSS usability. The research model of this empirical study establishes the relationship between the key usability factors and OSS usability from industrial perspective. In order to conduct …

Improvement Of Oss Usability: An Empirical Evaluation From Developers Perspective, Luiz Fernando Capretz, Arif Raza, Faheem Ahmed

Electrical and Computer Engineering Publications

No abstract provided.