Software Engineering Commons^™

Simplify Workflows: Ai As A Coding Companion, Tiffany Garrett

Library Scholarship

Artificial Intelligence is on everyone’s minds and has been the topic of the past two Matheson Lectures. But, what is the role of academic health sciences libraries? Moving the theoretical into practical, seven of our colleagues will present real-life case studies. What worked - what didn’t - what would they do differently?

This entry is from one of those case presentations on how a librarian at Roseman University of Health Sciences used AI to complete simple computer programming projects that optimized a few library workflows.

Large Language Models For Software Engineering: A Systematic Literature Review, Xinyi Hou, Yanjie Zhao, Yue Liu, Zhou Yang, Kailong Wang, Li Li, Xiapu Luo, David Lo, John Grundy, Haoyu Wang

Research Collection School Of Computing and Information Systems

Large Language Models (LLMs) have significantly impacted numerous domains, including Software Engineering (SE). Many recent publications have explored LLMs applied to various SE tasks. Nevertheless, a comprehensive understanding of the application, effects, and possible limitations of LLMs on SE is still in its early stages. To bridge this gap, we conducted a Systematic Literature Review (SLR) on LLM4SE, with a particular focus on understanding how LLMs can be exploited to optimize processes and outcomes. We selected and analyzed 395 research articles from January 2017 to January 2024 to answer four key Research Questions (RQs). In RQ1, we categorize different LLMs …

What Do We Know About Hugging Face? A Systematic Literature Review And Quantitative Validation Of Qualitative Claims, Jason Jones, Wenxin Jiang, Nicholas Synovic, George K. Thiruvathukal, James C. Davis

Computer Science: Faculty Publications and Other Works

Background: Collaborative Software Package Registries (SPRs) are an integral part of the software supply chain. Much engineering work synthesizes SPR package into applications. Prior research has examined SPRs for traditional software, such as NPM (JavaScript) and PyPI (Python). Pre-Trained Model (PTM) Registries are an emerging class of SPR of increasing importance, because they support the deep learning supply chain.
Aims: Recent empirical research has examined PTM registries in ways such as vulnerabilities, reuse processes, and evolution. However, no existing research synthesizes them to provide a systematic understanding of the current knowledge. Some of the existing research includes qualitative …

Fail Fast, Fail Small: Designing Resilient Systems For The Future Of Software Engineering, Jill Willard, James Hutson

Faculty Scholarship

The principles of "fail fast, fail small" have emerged as critical in modern software and system design. By planning for minor, manageable failures instead of catastrophic breakdowns, developers can ensure that systems degrade gracefully, maintaining functionality even when encountering issues. This article delves into strategies for designing resilient systems, beginning with the concept of slow degradation and distributed systems that prioritize core functions while allowing non-critical components to fail without significant user impact. The Netflix recommendation engine serves as a prime example of a system that continues to operate under failure conditions. Chaos engineering, a proactive methodology for stress-testing system …

Generative Ai In Software Engineering Must Be Human-Centered: The Copenhagen Manifesto, D. Russo, S. Van Berkel Baltes, Christoph Treude

Research Collection School Of Computing and Information Systems

The advent of Generative Artificial Intelligence—systems that can produce human-like content such as text, music, visual art, or source code—marks not only a significant leap for Artificial Intelligence (AI) but also a pivotal moment for software practitioners and researchers. The role of software engineering researchers and practitioners in adopting the technologies that shape our world is critical. Historically, the human aspects of developing software have been treated as secondary to more technical innovations. However, the emergence of Generative AI will simultaneously enhance human capabilities while surfacing complex ethical, social, legal, and technical challenges.While primarily aimed at software engineering (SE) researchers …

Documenting Ethical Considerations In Open Source Ai Models, Haoyu Gao, Mansooreh Zahedi, Christoph Treude, Sarita Rosenstock, Marc Cheong

Research Collection School Of Computing and Information Systems

Background: The development of AI-enabled software heavily depends on AI model documentation, such as model cards, due to different domain expertise between software engineers and model developers. From an ethical standpoint, AI model documentation conveys critical information on ethical considerations along with mitigation strategies for downstream developers to ensure the delivery of ethically compliant software. However, knowledge on such documentation practice remains scarce. Aims: The objective of our study is to investigate how developers document ethical aspects of open source AI models in practice, aiming at providing recommendations for future documentation endeavours. Method: We selected three sources of documentation on …

Digital Twin For Shelf Intelligence: Ai-Driven Inventory Management For Minimizing Food Waste, Charlotte Maples, Marvin Velazquez

College of Engineering Summer Undergraduate Research Program

This project aims to develop a solution for improving grocery store inventory management by leveraging AI-driven image recognition. Traditional inventory methods, which rely on manual counting or barcode scanning, are inefficient, labor-intensive, and prone to human error. Over an 8-week period, we designed and developed a basic iPad app capable of identifying specific types of fruit and automatically updating inventory records in real time. By utilizing the iPad’s camera and machine learning algorithms, the app demonstrates the potential to streamline inventory tracking, reduce manual labor, and improve accuracy in managing perishable goods. Future work will focus on expanding the app’s …

Enhancing Place-Based Interaction With Emotion Ai And Augmented Reality, Jake Maier, Ivan Martinez

College of Engineering Summer Undergraduate Research Program

This project explores the integration of augmented reality (AR) and Emotion AI technologies to enhance user experiences in physical environments. By seamlessly merging virtual elements with real-world contexts, we aim to deepen individuals’ interactions and perceptions of their surroundings. Leveraging AR technology enables users to access contextual information, engage with interactive content, and navigate spaces with heightened immersion and understanding. Additionally, Emotion AI enhances these experiences by detecting and responding to users’ emotional states, fostering personalized and emotionally resonant interactions. We aim to integrate digital content within physical environments using mixed-reality headsets equipped with eye-tracking capabilities and consumer-grade wireless EEG …

Promise And Peril Of Collaborative Code Generation Models : Balancing Effectiveness And Memorization, Zhi Chen, Lingxiao Jiang

Research Collection School Of Computing and Information Systems

In the rapidly evolving field of machine learning, training models with datasets from various locations and organizations presents significant challenges due to privacy and legal concerns. The exploration of effective collaborative training settings, which are capable of leveraging valuable knowledge from distributed and isolated datasets, is increasingly crucial. This study investigates key factors that impact the effectiveness of collaborative training methods in code next-token prediction, as well as the correctness and utility of the generated code, showing the promise of such methods. Additionally, we evaluate the memorization of different participant training data across various collaborative training settings, including centralized, federated, …

Exploring Conversations Between A Practitioner And A Person With Dementia, Kotaro Hara, Rosiana Natalie, Wei Soon Cheong, Jingjing Gu, Qianli Xu

Research Collection School Of Computing and Information Systems

In social service centers, practitioners engage in conversations with clients with dementia to facilitate their daily activities and provide support when they are distressed. However, the nature of the care demands the practitioner’s active engagement, which becomes difficult to deliver as the number of people who need care expands. Researchers have been investigating the efficacy of developing agents that assume conversational tasks to alleviate this work. To contribute to the future design of agents for caregiving, we collected and analyzed ten conversations between clients with mild dementia and practitioners who provide care. Our analyses of turn-taking dynamics and dialogue acts …

Interoperability In Deep Learning: A User Survey And Failure Analysis Of Onnx Model Converters, Purvish Jajal, Wenxin Jiang, Arav Tewari, Erik Kocinare, Joseph Woo, Anusha Sarraf, Yung-Hsiang Lu, George Thiruvathukal, James C. Davis

Computer Science: Faculty Publications and Other Works

Software engineers develop, fine-tune, and deploy deep learning (DL) models using a variety of development frameworks and runtime environments. DL model converters move models between frameworks and to runtime environments. Conversion errors compromise model quality and disrupt deployment. However, the failure characteristics of DL model converters are unknown, adding risk when using DL interoperability technologies. This paper analyzes failures in DL model converters. We survey software engineers about DL interoperability tools, use cases, and pain points (N=92). Then, we characterize failures in model converters associated with the main interoperability tool, ONNX (N=200 issues in PyTorch and TensorFlow). Finally, we formulate …

Quality Assurance In Software Engineering: A Journey Towards Explainable Automated Solutions, Ratnadira Widyasari

Dissertations and Theses Collection (Open Access)

In today's digital era, the pervasive influence of software on daily life underscores the necessity for high-quality and reliable systems. Software failures can result in substantial harm and financial losses, highlighting the pivotal role of Software Quality Assurance (SQA). While automated SQA techniques have evolved to aid developers in ensuring software quality, the necessity for explainability in these automated solutions has become equally important. For example, in automated fault localization, only identifying suspicious locations is insufficient; it is essential to provide reasoning on why these locations are suspicious. This dissertation presents a series of interconnected studies aimed at developing explainable …

How Effective Are They? Exploring Large Language Model Based Fuzz Driver Generation, Cen Zhang, Yaowen Zheng, Mingqiang Bai, Yeting Li, Wei Ma, Xiaofei Xie

Research Collection School Of Computing and Information Systems

Fuzz drivers are essential for library API fuzzing. However, automatically generating fuzz drivers is a complex task, as it demands the creation of high-quality, correct, and robust API usage code. An LLM-based (Large Language Model) approach for generating fuzz drivers is a promising area of research. Unlike traditional program analysis-based generators, this text-based approach is more generalized and capable of harnessing a variety of API usage information, resulting in code that is friendly for human readers. However, there is still a lack of understanding regarding the fundamental issues on this direction, such as its e ectiveness and potential challenges. To …

An Empirical Study Of Static Analysis Tools For Secure Code Review, Wachiraphan Charoenwet, Patanamon Thongtanunam, Van-Thuan Pham, Christoph Treude

Research Collection School Of Computing and Information Systems

Early identification of security issues in software development is vital to minimize their unanticipated impacts. Code review is a widely used manual analysis method that aims to uncover security issues along with other coding issues in software projects. While some studies suggest that automated static application security testing tools (SASTs) could enhance security issue identification, there is limited understanding of SAST’s practical effectiveness in supporting secure code review. Moreover, most SAST studies rely on synthetic or fully vulnerable versions of the subject program, which may not accurately represent real-world code changes in the code review process. To address this gap, …

Reless: A Framework For Assessing Safety In Deep Learning Systems, Nan Jia, Anita Raja, Raffi T. Khatchadourian

Publications and Research

Traditionally, software refactoring helps to improve a system's internal structure and enhance its non-functional features, such as reliability and run-time performance, while preserving external behavior including original program semantics. However, in the context of learning-enabled software systems (LESS), e.g., Machine Learning (ML) systems, it is unclear which portions of a software's semantics require preservation at the development phase. This is mainly because (a) the behavior of the LESS is not defined until run-time; and (b) the inherently iterative and non-deterministic nature of ML algorithms. Consequently, there is a knowledge gap in what refactoring truly means in the context of LESS …

Vysion Software, Isaias Hernandez-Dominguez Jr, Chander Luderman Miller

2024 Symposium

Vision loss presents significant challenges in daily life. Existing solutions for blind and visually impaired individuals are often limited in functionality, expensive, or complex to use. Vysion Software addresses this gap by developing a user-friendly, all-in-one AI companion app that provides features including text summarization, real-time audio descriptions, and AI-enhanced navigation. This project details the development plan, initial functionalities, and future vision for Vysion Software.

Peatmoss: A Dataset And Initial Analysis Of Pre-Trained Models In Open-Source Software, Wenxin Jiang, Jerin Yasmin, Jason Jones, Nicholas Synovic, Jiashen Kuo, Nathaniel Bielanski, Yuan Tian, George K. Thiruvathukal, James C. Davis

Computer Science: Faculty Publications and Other Works

The development and training of deep learning models have become increasingly costly and complex. Consequently, software engineers are adopting pre-trained models (PTMs) for their downstream applications. The dynamics of the PTM supply chain remain largely unexplored, signaling a clear need for structured datasets that document not only the metadata but also the subsequent applications of these models. Without such data, the MSR community cannot comprehensively understand the impact of PTM adoption and reuse. This paper presents the PeaTMOSS dataset, which comprises metadata for 281,638 PTMs and detailed snapshots for all PTMs with over 50 monthly downloads (14,296 PTMs), along with …

Generative Ai For Pull Request Descriptions: Adoption, Impact, And Developer Interventions, Tao Xiao, Hideaki Hata, Christoph Treude, Kenichi Matsumoto

Research Collection School Of Computing and Information Systems

GitHub's Copilot for Pull Requests (PRs) is a promising service aiming to automate various developer tasks related to PRs, such as generating summaries of changes or providing complete walkthroughs with links to the relevant code. As this innovative technology gains traction in the Open Source Software (OSS) community, it is crucial to examine its early adoption and its impact on the development process. Additionally, it offers a unique opportunity to observe how developers respond when they disagree with the generated content. In our study, we employ a mixed-methods approach, blending quantitative analysis with qualitative insights, to examine 18,256 PRs in …

How Do Preservice Teachers Learn To Teach Integrated Computational Thinking?: Evidence From Planning, Enactment, And Reflection, Rachael Dektor, Samuel Severance, Kip Téllez

Journal of Computer Science Integration

This study examines pre-service teachers’ (PSTs) beliefs and understandings about computational thinking (CT) integration and lesson implementation over time. Utilizing a design-based research approach, 3 PSTs led the co-design of integrated CT lessons with support from researchers and enacted these CT integrated lessons with K-5 students. All PSTs participated in a whole-group CT workshop and engaged in one-on-one lesson design sessions with a researcher. We utilized a grounded theory approach to qualitatively analyze pre-surveys, semi-structured interviews, and video data of three PSTs enacting their lessons. We found that PSTs’ initial beliefs about CT instruction – including the importance of it …

Henna Chatbot Capstone Review, Kobe Norcross

University Honors Theses

This thesis reviews the development of the Henna Chatbot, an AI-powered DEI consultant designed to provide personalized feedback to organizations. Sponsored by DEI consultant Arsh Haque, the project aims to address gaps in current DEI software, which often lacks team-specific feedback. The Henna Chatbot leverages GPT-3.5 Turbo to create an affordable SaaS platform where organizations can train Henna with their DEI values, and Henna will help organizations stay aligned with those values. The project spanned twenty weeks and was completed by a team of eight computer science students at Portland State University. The development process followed Agile methodologies, emphasizing effective …

Editorial Preview, Su-Cheng Haw

Journal of Informatics and Web Engineering

This editorial highlights all 18 papers in the June issue that deal with the practical aspects of Machine Learning (ML), Artificial Intelligence (AI), Data Mining (DM), the Internet of Things (IoT), Computer Vision, e-learning, and other topics in Computer Science. This issue also includes suggestions for several worthwhile works that deserve further research. With effective from our third volume first issue, we will be publishing triannually in February, June and October.

Facial Skin Analysis In Malaysians Using Yolov5: A Deep Learning Perspective, Ying Huey Gan, Shih Yin Ooi, Ying Han Pang, Yi Hong Tay, Quan Fong Yeo

Journal of Informatics and Web Engineering

Nowadays, people are more concerned about their skin conditions and are more willing to spend money and time on facial care routines. The beauty sector market is increasing, and more skin type readers are being created to help people determine their skin type. While various skin type readers are in the market, each is invented and tested abroad. Those skin type readers in the beauty market are not applied well on Malaysian skin. Therefore, this paper proposes a facial skin analysis system tailored primarily for Malaysian skin. This paper integrated object detection and deep learning algorithms in developing skin-type readers. …

Classroom Environment Analysis Via Internet Of Things, Kai-Yuan Tan, Kok-Why Ng, Kanesaraj Ramasamy

Journal of Informatics and Web Engineering

In this era of rapid technological advancement, the potential of the digital age has opened up numerous possibilities for our society. However, despite these advancements, traditional classrooms still lack the necessary technology to create an optimal learning environment for students. Consequently, students may struggle to effectively acquire knowledge within classrooms. This paper aims to conduct a classroom environment analysis using Internet of Things technology to gather data and uncover valuable insights. The proposed solution involves an embedded system for controlling and monitoring the classroom environment, as well as exporting historical data for further research. By ensuring accurate data collection, this …

Real Time 3d Internal Building Directory Map, Zi Yang Chia, Pey Yun Goh

Journal of Informatics and Web Engineering

Global Positioning System (GPS) is a famous technology around the world in identifying the real time precise location of any object with the assistance of satellites. The most common application of GPS is the use of outdoor maps. GPS offers efficient, scalable and cost-effective location services. However, this technology is not reliable when the position is in an indoor environment. The signal is very weak or totally lost due to signal attenuation and multipath effects. Among the indoor positioning technologies, WLAN is the most convenient and cost effective. In recent research, machine learning algorithms have become popular and utilized in …

Assessing The Efficiency Of Deep Learning Methods For Automated Vehicle Registration Recognition For University Entrance, Muhammad Syaqil Irsyad, Zarina Che Embi, Khairil Imran Bin Ghauth

Journal of Informatics and Web Engineering

With the ever-increasing number of vehicles on the road, a faster reliable security system for university entry is needed. This paper presents an approach for Automatic Number Plate Recognition (ANPR) using deep learning and PP-OCRv3. The proposed approach utilizes a pre-trained object detection model to locate license plates, extracts a single frame of the license plate, performs license plate recognition, applies pre-processing techniques, and employs PP-OCRv3 for text extraction in real time. The system was tested with Malaysian vehicle plates, and its accuracy and speed of detection were evaluated. The results show the system's potential to be easily adapted to …

Streamlining Dental Clinic Management For Effective Digitisation Productivity And Usability, Sin-Ban Ho, En-Yu Chew, Chuie-Hong Tan

Journal of Informatics and Web Engineering

Oral health is an integral part of overall health, and poor oral hygiene can lead to a variety of health problems. Modern oral care has greatly improved our quality of life, but the increasing demand for routine dental checkups and treatments calls for improved systems for managing patient records and appointments. While technology has significantly enhanced the efficacy and experience of dental care, many dental clinics still rely on paper records to record the patient’s oral condition, but these are not easily accessible to the patients for viewing. This study aims to address the issue by developing a Dental Clinic …

Knowledge-Based Word Tokenization System For Urdu, Asif Khan, Khairullah Khan, Wahab Khan, Sadiq Nawaz Khan, Rafiul Haq

Journal of Informatics and Web Engineering

Word tokenization, a foundational step in natural language processing (NLP), is critical for tasks like part-of-speech tagging, named entity recognition, and parsing, as well as various independent NLP applications. In our tech-driven era, the exponential growth of textual data on the World Wide Web demands sophisticated tools for effective processing. Urdu, spoken widely across the globe, is experiencing a surge in, presents unique challenges due to its distinct writing style, the absence of capitalization features, and the prevalence of compound words. This study introduces a novel knowledge-based word tokenization system tailored for Urdu. Central to this system is a maximum …

Sentiment Analysis In Social Media: A Case Study Of Hike In University School Fees In Selected Nigerian Universities, Abdulahi Olarewaju Aremu, Isah Muhammad

Journal of Informatics and Web Engineering

Faced with escalating operational costs and government disinvestment, Nigerian public universities are implementing tuition fee increases to maintain institutional functionality. This necessary fiscal measure comes in the wake of 2022 industrial action, which exacerbated pre-existing financial strain through extended work stoppages and potentially higher costs associated with resuming activities, while leaving unaddressed the longstanding demands of academics for improved welfare and working conditions. The court-mandated resumption of academic activities without resolution of these core issues further strained university finances, leading to a significant increase in tuition fees. Using VADER, this study investigated social media sentiments related to the increase in …

A Bilingual Immersive Environment For Kids Learning, Tufail Ahmad, Arifa Bibi, Umar Farooq, Ihsan Rabbi

Journal of Informatics and Web Engineering

Virtual Worlds (VWs) provide an improved learning experience as they have an increased level of user participation and technological involvement. They avoid content duplication and following a set of predefined rules to overcome the issues in game environments. VWs are coherent, persistent and collaborative social places that believe in realism through immersion. Client viewer software is used for user immersion in the form of avatars and in-world content creation in these environments. This work presents a simple bi-lingual environment developed for kids’ learning in English and Urdu using the well-known VW development framework called OpenSimulator (OpSim). It used Firestorm viewer …

Unveiling The Efficacy Of Ai-Based Algorithms In Phishing Attack Detection, Tajamul Shahzad, Kashif Aman

Journal of Informatics and Web Engineering

Phishing poses a significant challenge in an ever-evolving world. The increased usage of the Internet has resulted in the emergence of a different kind of theft referred to as cybercrime. The term cybercrime describes the act of invading privacy and illegitimately obtaining personal information using digital platform. Primarily an approach named phishing is employed, which involves the use of spoof emails or bogus websites by the attackers to get the victim's personal information like their account credentials, debit, or credit card’s number, etc. To give the brief knowledge of phishing attacks and their types of the objective of this work …