Physical Sciences and Mathematics Commons^™

A Computer Vision Solution To Cross-Cultural Food Image Classification And Nutrition Logging​, Rohan Sethi, George K. Thiruvathukal

Computer Science: Faculty Publications and Other Works

The US is a culturally and ethnically diverse country, and with this diversity comes a myriad of cuisines and eating habits that expand well beyond that of western culture. Each of these meals have their own good and bad effects when it comes to the nutritional value and its potential impact on human health. Thus, there is a greater need for people to be able to access the nutritional profile of their diverse daily meals and better manage their health. A revolutionary solution to democratize food image classification and nutritional logging is using deep learning to extract that information from …

Terry Riley's "In C" For Mobile Ensemble, David B. Wetzel, Griffin Moe, George K. Thiruvathukal

Computer Science: Faculty Publications and Other Works

This workshop presents a mobile-friendly Web Audio application for a “technology ensemble play-along” of Terry Riley’s 1964 composition In C. Attendees will join in a reading of In C using available web-enabled devices as musical instruments. We hope to demonstrate an accessible music-technology experience that relies on face-to-face interaction within a shared space. In this all-electronic implementation, no special musical or technical expertise is required.

Accepted for presentation and publication at WAC 2024.

Emoji Use In Social Media Posts: Relationships With Personality Traits And Word Usage, Shelia Kennison, Kameryn Fritz, Maria Andrea Hurtado Morales, Eric Chan-Tin

Computer Science: Faculty Publications and Other Works

Prior research has demonstrated relationships between personality traits of social media users and the language used in their posts. Few studies have examined whether there are relationships between personality traits of users and how they use emojis in their social media posts. Emojis are digital pictographs used to express ideas and emotions. There are thousands of emojis, which depict faces with expressions, objects, animals, and activities. We conducted a study with two samples (n = 76 and n = 245) in which we examined how emoji use on X (formerly Twitter) related to users’ personality traits and language use …

Actionpoint: An App To Combat Cyberbullying By Strengthening Parent-Teen Relationships, Maddie Juarez, Natali Barragan, Deborah Hall, George K. Thiruvathukal, Yasin Silva

Computer Science: Faculty Publications and Other Works

No abstract provided.

Optimized Uncertainty Estimation For Vision Transformers: Enhancing Adversarial Robustness And Performance Using Selective Classification, Erik Pautsch, John Li, Silvio Rizzi, George K. Thiruvathukal, Maria Pantoja

Computer Science: Faculty Publications and Other Works

Deep Learning models often exhibit undue confidence when encountering out-of-distribution (OOD) inputs, misclassifying with high confidence. The ideal outcome, in these cases, would be an "I do not know" verdict. We enhance the trustworthiness of our models through selective classification, allowing the model to abstain from making predictions when facing uncertainty. Rather than a singular prediction, the model offers a prediction distribution, enabling users to gauge the model’s trustworthiness and determine the need for human intervention. We assess uncertainty in two baseline models: a Convolutional Neural Network (CNN) and a Vision Transformer (ViT). By leveraging these uncertainty values, we minimize …

Optimizing Uncertainty Quantification Of Vision Transformers In Deep Learning On Novel Ai Architectures, Erik Pautsch, John Li, Silvio Rizzi, George K. Thiruvathukal, Maria Pantoja

Computer Science: Faculty Publications and Other Works

Deep Learning (DL) methods have shown substantial efficacy in computer vision (CV) and natural language processing (NLP). Despite their proficiency, the inconsistency in input data distributions can compromise prediction reliability. This study mitigates this issue by introducing uncertainty evaluations in DL models, thereby enhancing dependability through a distribution of predictions. Our focus lies on the Vision Transformer (ViT), a DL model that harmonizes both local and global behavior. We conduct extensive experiments on the ImageNet-1K dataset, a vast resource with over a million images across 1,000 categories. ViTs, while competitive, are vulnerable to adversarial attacks, making uncertainty estimation crucial for …

Peatmoss: Mining Pre-Trained Models In Open-Source Software, Wenxin Jiang, Jason Jones, Jerin Yasmin, Nicholas Synovic, Rajiv Sashti, Sophie Chen, George K. Thiruvathukal, Yuan Tian, James C. Davis

Computer Science: Faculty Publications and Other Works

Developing and training deep learning models is expensive, so software engineers have begun to reuse pre-trained deep learning models (PTMs) and fine-tune them for downstream tasks. Despite the widespread use of PTMs, we know little about the corresponding software engineering behaviors and challenges. To enable the study of software engineering with PTMs, we present the PeaTMOSS dataset: Pre-Trained Models in Open-Source Software. PeaTMOSS has three parts: a snapshot of (1) 281,638 PTMs, (2) 27,270 open-source software repositories that use PTMs, and (3) a mapping between PTMs and the projects that use them. We challenge PeaTMOSS miners to discover software engineering …

Tree-Based Unidirectional Neural Networks For Low-Power Computer Vision, Abhinav Goel, Caleb Tung, Nick Eliopoulos, Amy Wang, Jamie C. Davis, George K. Thiruvathukal, Yung-Hisang Lu

Computer Science: Faculty Publications and Other Works

This article describes the novel Tree-based Unidirectional Neural Network (TRUNK) architecture. This architecture improves computer vision efficiency by using a hierarchy of multiple shallow Convolutional Neural Networks (CNNs), instead of a single very deep CNN. We demonstrate this architecture’s versatility in performing different computer vision tasks efficiently on embedded devices. Across various computer vision tasks, the TRUNK architecture consumes 65% less energy and requires 50% less memory than representative low-power CNN architectures, e.g., MobileNet v2, when deployed on the NVIDIA Jetson Nano.

Assessing The Impact Of A Csforall Research-Practice Partnership Using The Prosper Framework: A Case Study Of The Chicago Alliance For Equity In Computer Science (Cafécs), Erin Henrick, Steven Mcgee, Ronald I. Greenberg, Dale Reed, Don Yanek, Lucia Dettori, Haley Williamson

Computer Science: Faculty Publications and Other Works

The Chicago Alliance for Equity in Computer Science (CAFÉCS) Research Practice Partnership (RPP) has been working for more than a decade towards their mission to engage in research and development that enables Chicago Public Schools (CPS) to ensure that all students in Chicago participate in engaging, relevant, and rigorous computing experiences, increase opportunities for all students to pursue computing pathways and prepare all students for the future of work. The partnership engaged in an iterative design process to develop a framework for understanding the areas of RPP impact on a district. This paper applies the PROSPER framework to the CAFÉCS …

Conversations With Chatgpt About C Programming: An Ongoing Study, James C. Davis, Yung-Hsiang Lu, George K. Thiruvathukal

Computer Science: Faculty Publications and Other Works

AI (Artificial Intelligence) Generative Models have attracted great attention in recent years. Generative models can be used to create new articles, visual arts, music composition, even computer programs from English specifications. Among all generative models, ChatGPT is becoming one of the most well-known since its public announcement in November 2022. GPT means {\it Generative Pre-trained Transformer}. ChatGPT is an online program that can interact with human users in text formats and is able to answer questions in many topics, including computer programming. Many computer programmers, including students and professionals, are considering the use of ChatGPT as an aid. The quality …

Observing Human Mobility Internationally During Covid-19, Shane Allcroft, Mohammed Metwaly, Zachery Berg, Isha Ghodgaonkar, Fischer Bordwell, Xinxin Zhao, Xinglei Liu, Jiahao Xu, Subhankar Chakraborty, Vishnu Banna, Akhil Chinnakotla, Abhinav Goel, Caleb Tung, Gore Kao, Wei Zakharov, David A. Shoham, George K. Thiruvathukal, Yung-Hsiang Lu

Computer Science: Faculty Publications and Other Works

This article analyzes visual data captured from five countries and three U.S. states to evaluate the effectiveness of lockdown policies for reducing the spread of COVID-19. The main challenge is the scale: nearly six million images are analyzed to observe how people respond to the policy changes.

Poster: Userland Containers For Mobile Systems, Isaac Ahlgren, Victor Rakotondranoro, Yasin N. Silva, Eric Chan-Tin, George K. Thiruvathukal, Neil Klingensmith

Computer Science: Faculty Publications and Other Works

Mobile platforms are not rising to their potential as ubiquitous computers, in large part because of the constraints we impose on their apps in the name of security. Mobile operating systems have long struggled with the challenge of isolating untrusted apps. In pursuit of a secure runtime environment, Android and iOS isolate apps inside a gulag of platform-imposed programming languages and runtime libraries, leaving few design decisions to the application developers. These thick layers of custom software eschew app portability and maintainability, as development teams must continually tweak their apps to support modifications to the OS's runtime libraries. Nonstandard and …

Snapshot Metrics Are Not Enough: Analyzing Software Repositories With Longitudinal Metrics, Nicholas Synovic, Matt Hyattt, Rohan Sethi, Sohini Thota, Shilpika, Allan J. Miller, Wenxin Jiang, Emmanuel S. Amobi, Austin Pinderski, Konstantin Läufer, Nicholas J. Hayward, Neil Klingensmith, James C. Davis, George K. Thiruvathukal

Computer Science: Faculty Publications and Other Works

Software metrics capture information about software development processes and products. These metrics support decision-making, e.g., in team management or dependency selection. However, existing metrics tools measure only a snapshot of a software project. Little attention has been given to enabling engineers to reason about metric trends over time -- longitudinal metrics that give insight about process, not just product. In this work, we present PRiME (PRocess MEtrics), a tool for computing and visualizing process metrics. The currently-supported metrics include productivity, issue density, issue spoilage, and bus factor. We illustrate the value of longitudinal data and conclude with a research agenda. …

Evolution Of Winning Solutions In The 2021 Low-Power Computer Vision Challenge, Xiao Hu, Ziteng Jiao, Ayden Kocher, Zhenyu Wu, Junjie Liu, James C. Davis, George K. Thiruvathukal, Yung-Hsiang Lu

Computer Science: Faculty Publications and Other Works

Mobile and embedded devices are becoming ubiquitous. Applications such as rescue with autonomous robots and event analysis on traffic cameras rely on devices with limited power supply and computational sources. Thus, the demand for efficient computer vision algorithms increases. Since 2015, we have organized the IEEE Low-Power Computer Vision Challenge to advance the state of the art in low-power computer vision. We describe the competition organizing details including the challenge design, the reference solution, the dataset, the referee system, and the evolution of the solutions from two winning teams. We examine the winning teams’ development patterns and design decisions, focusing …

College Teaching And Ai, Leo Irakliotis

Computer Science: Faculty Publications and Other Works

Artificial Intelligence will reshape the way we assess student learning in ways that no one has prepared us for.

Chicago Alliance For Equity In Computer Science, Steven Mcgee, Lucia Dettori, Ronald I. Greenberg, Andrew M. Rasmussen, Dale F. Reed, Don Yanek

Computer Science: Faculty Publications and Other Works

Each year, about 14,000 Chicago Public Schools (CPS) students graduate with one year of high school computer science (CS) in fulfillment of the district’s CS graduation requirement. This accomplishment was the culmination of a decade of work by the Chicago Alliance for Equity in Computer Science (CAFÉCS), which includes CPS teachers and administrators, university CS faculty, and educational researchers. CAFÉCS research indicates that CPS significantly increased the capacity of schools to offer the Exploring Computer Science (ECS) introductory course, resulting in a rapid, equitable increase in students’ participation in CS. Making CS mandatory did not negatively impact performance in ECS. …

Unoapi: Balancing Performance, Portability, And Productivity (P3) In Hpc Education, Konstantin Laufer, George K. Thiruvathukal

Computer Science: Faculty Publications and Other Works

oneAPI is a major initiative by Intel aimed at making it easier to program heterogeneous architectures used in high-performance computing using a unified application programming interface (API). While raising the abstraction level via a unified API represents a promising step for the current generation of students and practitioners to embrace high- performance computing, we argue that a curriculum of well- developed software engineering methods and well-crafted exem- plars will be necessary to ensure interest by this audience and those who teach them. We aim to bridge the gap by developing a curriculum—codenamed UnoAPI—that takes a more holistic approach by looking …

An Empirical Study Of Artifacts And Security Risks In The Pre-Trained Model Supply Chain, Wenxin Jiang, Nicholas Synovic, Rohan Sethi, Aryan Indarapu, Matt Hyattt, Taylor R. Schorlemmer, George K. Thiruvathukal, James C. Davis

Computer Science: Faculty Publications and Other Works

Deep neural networks achieve state-of-the-art performance on many tasks, but require increasingly complex architectures and costly training procedures. Engineers can reduce costs by reusing a pre-trained model (PTM) and fine-tuning it for their own tasks. To facilitate software reuse, engineers collaborate around model hubs, collections of PTMs and datasets organized by problem domain. Although model hubs are now comparable in popularity and size to other software ecosystems, the associated PTM supply chain has not yet been examined from a software engineering perspective.

We present an empirical study of artifacts and security features in 8 model hubs. We indicate the potential …

Sila: A System For Scientific Image Analysis, Daniel Moreira, João Phillipe Cardenuto, Ruiting Shao, Sriram Baireddy, Davide Cozzolino, Diego Gragnaniello, Wael Abd-Almageed, Paolo Bestagini, Stefano Tubaro, Anderson Rocha, Walter Scheirer, Luisa Verdoliva, Edward Delp

Computer Science: Faculty Publications and Other Works

A great deal of the images found in scientific publications are retouched, reused, or composed to enhance the quality of the presentation. In most instances, these edits are benign and help the reader better understand the material in a paper. However, some edits are instances of scientific misconduct and undermine the integrity of the presented research. Determining the legitimacy of edits made to scientific images is an open problem that no current technology can perform satisfactorily in a fully automated fashion. It thus remains up to human experts to inspect images as part of the peer-review process. Nonetheless, image analysis …

Directed Acyclic Graph-Based Neural Networks For Tunable Low-Power Computer Vision, Abhinav Goel, Caleb Tung, Nick Eliopoulos, Xiao Hu, George K. Thiruvathukal, James C. Davis, Yung-Hisang Lu

Computer Science: Faculty Publications and Other Works

Processing visual data on mobile devices has many applications, e.g., emergency response and tracking. State-of-the-art computer vision techniques rely on large Deep Neural Networks (DNNs) that are usually too power-hungry to be deployed on resource-constrained edge devices. Many techniques improve DNN efficiency of DNNs by compromising accuracy. However, the accuracy and efficiency of these techniques cannot be adapted for diverse edge applications with different hardware constraints and accuracy requirements. This paper demonstrates that a recent, efficient tree-based DNN architecture, called the hierarchical DNN, can be converted into a Directed Acyclic Graph-based (DAG) architecture to provide tunable accuracy-efficiency tradeoff options. We …

Finding Approximate Pythagorean Triples (And Applications To Lego Robot Building), Ronald I. Greenberg, Matthew Fahrenbacher, George K. Thiruvathukal

Computer Science: Faculty Publications and Other Works

This assignment combines programming and data analysis to determine good combinations of side lengths that approximately satisfy the Pythagorean Theorem for right triangles. This can be a standalone exercise using a wide variety of programming languages, but the results are useful for determining good ways to assemble LEGO pieces in robot construction, so the exercise can serve to integrate three different units of the Exploring Computer Science high school curriculum: "Programming", "Computing and Data Analysis", and "Robotics". Sample assignment handouts are provided for both Scratch and Java programmers. Ideas for several variants of the assignment are also provided.

Using Magic To Teach Computer Programming, Dale F. Reed, Ronald I. Greenberg

Computer Science: Faculty Publications and Other Works

Magic can be used in project-based instruction to motivate students and provide a meaningful context for learning computer programming. This work describes several magic programs of the “Choose a Number” and “Pick a Card” varieties, making connections to underlying computing concepts.

Magic tricks presented as demonstrations and programming assignments elicit wonder and captivate students’ attention, so that students want to understand and replicate the work to show it to friends and family members. Capturing student interest and curiosity motivates them to learn the underlying programming concepts.

Two “Choose a Number” programs are shown where the computer is able to identify …

Are You Really Muted?: A Privacy Analysis Of Mute Buttons In Video Conferencing Apps, Yucheng Yang, Jack West, George K. Thiruvathukal, Neil Klingensmith, Kassem Fawaz

Computer Science: Faculty Publications and Other Works

In the post-pandemic era, video conferencing apps (VCAs) have converted previously private spaces — bedrooms, living rooms, and kitchens — into semi-public extensions of the office. And for the most part, users have accepted these apps in their personal space, without much thought about the permission models that govern the use of their personal data during meetings. While access to a device’s video camera is carefully controlled, little has been done to ensure the same level of privacy for accessing the microphone. In this work, we ask the question: what happens to the microphone data when a user clicks the …

Irrelevant Pixels Are Everywhere: Find And Exclude Them For More Efficient Computer Vision, Caleb Tung, Abhinav Goel, Xiao Hu, Nick Eliopoulos, Emmanuel Amobi, George K. Thiruvathukal, Vipin Chaudhary, Yung-Hisang Lu

Computer Science: Faculty Publications and Other Works

Computer vision is often performed using Convolutional Neural Networks (CNNs). CNNs are compute-intensive and challenging to deploy on power-constrained systems such as mobile and Internet-of-Things (IoT) devices. CNNs are compute-intensive because they indiscriminately compute many features on all pixels of the input image. We observe that, given a computer vision task, images often contain pixels that are irrelevant to the task. For example, if the task is looking for cars, pixels in the sky are not very useful. Therefore, we propose that a CNN be modified to only operate on relevant pixels to save computation and energy. We propose a …

Phishing For Fun, Madeline Moran, Anna Hart, Loretta Stalans, Eric Chan-Tin, Shelia Kennison

Computer Science: Faculty Publications and Other Works

Perform a phishing experiment to see how many people fall victim. This study was approved by the Loyola IRB

Establishing Trust In Vehicle-To-Vehicle Coordination: A Sensor Fusion Approach, Jakob Veselsky, Jack West, Isaac Ahlgren, George K. Thiruvathukal, Neil Klingensmith, Abhinav Goel, Wenxin Jiang, James C. Davis, Kyuin Lee, Younghyun Kim

Computer Science: Faculty Publications and Other Works

Autonomous vehicles (AVs) use diverse sensors to understand their surroundings as they continually make safety- critical decisions. However, establishing trust with other AVs is a key prerequisite because safety-critical decisions cannot be made based on data shared from untrusted sources. Existing protocols require an infrastructure network connection and a third-party root of trust to establish a secure channel, which are not always available.

In this paper, we propose a sensor-fusion approach for mobile trust establishment, which combines GPS and visual data. The combined data forms evidence that one vehicle is nearby another, which is a strong indication that it is …

Forensic Analysis Of Synthetically Generated Western Blot Images, Sara Mandelli, Davide Cozzolino, Edoardo Cannas, João Phillipe Cardenuto, Daniel Moreira, Paolo Bestagini, Walter Scheirer, Anderson Rocha, Luisa Verdoliva, Stefano Tubaro, Edward Delp

Computer Science: Faculty Publications and Other Works

The widespread diffusion of synthetically generated content is a serious threat that needs urgent countermeasures. As a matter of fact, the generation of synthetic content is not restricted to multimedia data like videos, photographs or audio sequences, but covers a significantly vast area that can include biological images as well, such as western blot and microscopic images. In this paper, we focus on the detection of synthetically generated western blot images. These images are largely explored in the biomedical literature and it has been already shown they can be easily counterfeited with few hopes to spot manipulations by visual inspection …

Image Provenance Analysis, Daniel Moreira, William Theisen, Walter Scheirer, Aparna Bharati, Joel Brogan, Anderson Rocha

Computer Science: Faculty Publications and Other Works

The literature of multimedia forensics is mainly dedicated to the analysis of single assets (such as sole image or video files), aiming at individually assessing their authenticity. Different from this, image provenance analysis is devoted to the joint examination of multiple assets, intending to ascertain their history of edits, by evaluating pairwise relationships. Each relationship, thus, expresses the probability of one asset giving rise to the other, through either global or local operations, such as data compression, resizing, color-space modifications, content blurring, and content splicing. The principled combination of these relationships unveils the provenance of the assets, also constituting an …

Aerokey: Using Ambient Electromagnetic Radiation For Secure And Usable Wireless Device Authentication, Kyuin Lee, Yucheng Yang, Omkar Prabhune, Aishwarya Lekshmi Chithra, Jack West, Kassem Fawaz, Neil Klingensmith, Uman Banerjee, Younghyun Kim

Computer Science: Faculty Publications and Other Works

Wireless connectivity is becoming common in increasingly diverse personal devices, enabling various interoperation- and Internet-based applications and services. More and more interconnected devices are simultaneously operated by a single user with short-lived connections, making usable device authentication methods imperative to ensure both high security and seamless user experience. Unfortunately, current authentication methods that heavily require human involvement, in addition to form factor and mobility constraints, make this balance hard to achieve, often forcing users to choose between security and convenience. In this work, we present a novel over-the-air device authentication scheme named AEROKEY that achieves both high security and high …

Establishing Trust In Vehicle-To-Vehicle Coordination: A Sensor Fusion Approach, Jakob Veselsky, Jack West, Isaac Ahlgren, George K. Thiruvathukal, Neil Klingensmith, Abhinav Goel, Wenxin Jiang, James C. Davis, Kyuin Lee, Younghyun Kim

Computer Science: Faculty Publications and Other Works

As we add more autonomous and semi-autonomous vehicles (AVs) to our roads, their effects on passenger and pedestrian safety are becoming more important. Despite extensive testing, AVs do not always identify roadway hazards. Failures in object recognition components have already led to several fatal collisions, e.g. as a result of faults in sensors, software, or vantage point. Although a particular AV may fail, there is an untapped pool of information held by other AVs in the vicinity that could be used to identify roadway hazards before they present a safety threat.