Computer Sciences Commons^™

Compositional Prompt Tuning With Motion Cues For Open-Vocabulary Video Relation Detection, Kaifeng Gao, Long Chen, Hanwang Zhang, Jun Xiao, Qianru Sun

Research Collection School Of Computing and Information Systems

Prompt tuning with large-scale pretrained vision-language models empowers open-vocabulary prediction trained on limited base categories, e.g., object classification and detection. In this paper, we propose compositional prompt tuning with motion cues: an extended prompt tuning paradigm for compositional predictions of video data. In particular, we present Relation Prompt (RePro) for Open-vocabulary Video Visual Relation Detection (Open-VidVRD), where conventional prompt tuning is easily biased to certain subject-object combinations and motion patterns. To this end, RePro addresses the two technical challenges of Open-VidVRD: 1) the prompt tokens should respect the two different semantic roles of subject and object, and 2) the tuning …

Diffseer: Difference-Based Dynamic Weighted Graph Visualization, Xiaolin Wen, Yong Wang, Meixuan Wu, Fengjie Wang, Xuanwu Yue, Qiaomu Shen, Yuxin Ma, Min Zhu

Research Collection School Of Computing and Information Systems

Existing dynamic weighted graph visualization approaches rely on users’ mental comparison to perceive temporal evolution of dynamic weighted graphs, hindering users from effectively analyzing changes across multiple timeslices. We propose DiffSeer, a novel approach for dynamic weighted graph visualization by explicitly visualizing the differences of graph structures (e.g., edge weight differences) between adjacent timeslices. Specifically, we present a novel nested matrix design that overviews the graph structure differences over a time period as well as shows graph structure details in the timeslices of user interest. By collectively considering the overall temporal evolution and structure details in each timeslice, an optimization-based …

Sprout: Using A Garden Metaphor To Visualize And Support Customizable And Collaborative Health Tracking, Pape Sow Traoré

Dartmouth College Master’s Theses

Self-tracking tools have become increasingly popular, especially with the advent of wearable technology and smartphone applications. However, traditional tracking tools often display data in a quantitative format that can be overwhelming and cause users to abandon their tracking efforts. Additionally, these tools typically provide a generic user experience and are designed from a single-user perspective, lacking external support. To overcome these limitations, we develop Sprout, a mobile data-tracking application that offers a more qualitative, customizable, and collaborative experience for health monitoring and management. Sprout uses a garden metaphor to visually represent health information and allows users to tailor their …

Using Immersive Technology To Improve Mechanical Design: Use Cases, A Review Of Current Technology, And An Experiment In Requirement Elicitation In Virtual Reality, William Hawthorne

All Theses

There has been a growing trend in the use of immersive technologies within engineering design. This research is focused on understanding how Virtual Reality (VR) technologies support design reviews. Modern tools including virtual reality hardware and software were tested for current capabilities and challenges. In a review of literature, two key gaps are identified: the rapid advancements of VR and AR technology and limited formal studies to determine the costs and benefits of immersive technology within engineering design reviews. This research has resulted in three key outcomes. First, use cases of immersive reality technologies are identified, through the lens of …

Artificial: A Study On The Use Of Artificial Intelligence In Art, Hayden Ernst

Theses/Capstones/Creative Projects

In the past three to five years there have been significant improvements made in AI due to improvements in computing capacity, the collection and use of big data, and an increase in public interest and funding for research. Programs such as ChatGPT, DALL•E, and Midjourney have also gained tremendous popularity in a relatively short amount of time. This led me to this project in which I aimed to gain a deeper understanding of these art generator AI and where they fit into art as a whole. My goal was to give recommendations to museums and exhibits in Omaha on what …

Iot Health Devices: Exploring Security Risks In The Connected Landscape, Abasi-Amefon Obot Affia, Hilary Finch, Woosub Jung, Issah Abubakari Samori, Lucas Potter, Xavier-Lewis Palmer

School of Cybersecurity Faculty Publications

The concept of the Internet of Things (IoT) spans decades, and the same can be said for its inclusion in healthcare. The IoT is an attractive target in medicine; it offers considerable potential in expanding care. However, the application of the IoT in healthcare is fraught with an array of challenges, and also, through it, numerous vulnerabilities that translate to wider attack surfaces and deeper degrees of damage possible to both consumers and their confidence within health systems, as a result of patient-specific data being available to access. Further, when IoT health devices (IoTHDs) are developed, a diverse range of …

Procedural City Generation With Combined Architectures For Real-Time Visualization, Griffin Poyck

All Theses

The work and research of this paper sought to build upon traditional city generation and simulation in creating a tool that both realistically simulates cities and their prominent features and also creates aesthetic and artistically rich cities using assets that combine several contemporary or near contemporary architectural styles. The major city features simulated are the surrounding terrain, road networks, individual buildings, and building placement. The tools used to both create and integrate these features were created in Houdini with Unreal Engine 5 as the intended final destination. This research was influenced by the city, town, and road networking of Ghost …

Mando-Hgt: Heterogeneous Graph Transformers For Smart Contract Vulnerability Detection, Huu Hoang Nguyen, Nhat Minh Nguyen, Chunyao Xie, Zahra Ahmadi, Daniel Kudendo, Thanh-Nam Doan, Lingxiao Jiang

Research Collection School Of Computing and Information Systems

Smart contracts in blockchains have been increasingly used for high-value business applications. It is essential to check smart contracts' reliability before and after deployment. Although various program analysis and deep learning techniques have been proposed to detect vulnerabilities in either Ethereum smart contract source code or bytecode, their detection accuracy and scalability are still limited. This paper presents a novel framework named MANDO-HGT for detecting smart contract vulnerabilities. Given Ethereum smart contracts, either in source code or bytecode form, and vulnerable or clean, MANDO-HGT custom-builds heterogeneous contract graphs (HCGs) to represent control-flow and/or function-call information of the code. It then …

Document Graph Representation Learning, Ce Zhang

Dissertations and Theses Collection (Open Access)

Much of the data on the Web can be represented in a graph structure, ranging from social and biological to academic and Web page graphs, etc. Graph analysis recently attracts escalating research attention due to its importance and wide applicability. Diverse problems could be formulated as graph tasks, such as text classification and information retrieval. As the primary information is the inherent structure of the graph itself, one promising direction known as the graph representation learning problem is to learn the representation of each node, which could in turn fuel tasks such as node classification, node clustering, and link prediction. …

Liloc: Enabling Precise 3d Localization In Dynamic Indoor Environments Using Lidars, Darshana Rathnayake, Meera Radhakrishnan, Inseok Hwang, Archan Misra

Research Collection School Of Computing and Information Systems

We present LiLoc, a system for precise 3D localization and tracking of mobile IoT devices (e.g., robots) in indoor environments using multi-perspective LiDAR sensing. The key differentiators in our work are: (a) First, unlike traditional localization approaches, our approach is robust to dynamically changing environmental conditions (e.g., varying crowd levels, object placement/layout changes); (b) Second, unlike prior work on visual and 3D SLAM, LiLoc is not dependent on a pre-built static map of the environment and instead works by utilizing dynamically updated point clouds captured from both infrastructural-mounted LiDARs and LiDARs equipped on individual mobile IoT devices. To achieve fine-grained, …

Graph Neural Point Process For Temporal Interaction Prediction, Wenwen Xia, Yuchen Li, Shengdong Li

Research Collection School Of Computing and Information Systems

Temporal graphs are ubiquitous data structures in many scenarios, including social networks, user-item interaction networks, etc. In this paper, we focus on predicting the exact time of the next interaction, given a node pair on a temporal graph. This novel problem can support interesting applications, such as time-sensitive items recommendation, congestion prediction on road networks, and many others. We present Graph Neural Point Process (GNPP) to tackle this problem. GNPP relies on the graph neural message passing and the temporal point process framework. Most previous graph neural models only utilize the chronological order of observed events and ignore exact timestamps. …

Chronos: Time-Aware Zero-Shot Identification Of Libraries From Vulnerability Reports, Yunbo Lyu, Thanh Le Cong, Hong Jin Kang, Ratnadira Widyasari, Zhipeng Zhao, Xuan-Bach Dinh Le, Ming Li, David Lo

Research Collection School Of Computing and Information Systems

Tools that alert developers about library vulnerabilities depend on accurate, up-to-date vulnerability databases which are maintained by security researchers. These databases record the libraries related to each vulnerability. However, the vulnerability reports may not explicitly list every library and human analysis is required to determine all the relevant libraries. Human analysis may be slow and expensive, which motivates the need for automated approaches. Researchers and practitioners have proposed to automatically identify libraries from vulnerability reports using extreme multi-label learning (XML). While state-of-the-art XML techniques showed promising performance, their experimental settings do not practically fit what happens in reality. Previous studies …

Immersive Learning Environments For Computer Science Education, Dillon Buchanan

Electronic Theses and Dissertations

This master's thesis explores the effectiveness of an educational intervention using an interactive notebook to support and supplement instruction in a foundational-level programming course. A quantitative, quasi-experimental group comparison method was employed, where students were placed into either a control or a treatment group. Data was collected from assignment and final grades, as well as self-reported time spent using the notebook. Independent t-tests and correlation were used for data analysis. Results were inconclusive but did indicate that the intervention had a possible effect. Further studies may explore better efficacy, implementation, and satisfaction of interactive notebooks across a larger population and …

Accessible Virtual Reality For Older Adults, Aaron Gluck

All Dissertations

Virtual reality (VR) has grown significantly since the commercial release of the Oculus Rift in March 2016. This growth accelerated during the COVID-19 pandemic, revolutionizing how individuals and businesses work, socialize, exercise, and stay entertained. However, commercial VR systems are not designed to be accessible to people with disabilities. While researchers have explored aspects of VR accessibility for people with disabilities, there is minimal research on accessible VR for older adults. Older adults (65+) self-report the highest rate of disabilities which may result in VR accessibility barriers, making this the ideal group to study the accessibility of VR.

This dissertation …

Lpt: Long-Tailed Prompt Tuning For Image Classification, Bowen Dong, Pan Zhou, Shuicheng Yan, Wangmeng Zuo

Research Collection School Of Computing and Information Systems

For long-tailed classification tasks, most works often pretrain a big model on a large-scale (unlabeled) dataset, and then fine-tune the whole pretrained model for adapting to long-tailed data. Though promising, fine-tuning the whole pretrained model tends to suffer from high cost in computation and deployment of different models for different tasks, as well as weakened generalization capability for overfitting to certain features of long-tailed data. To alleviate these issues, we propose an effective Long-tailed Prompt Tuning (LPT) method for long-tailed classification tasks. LPT introduces several trainable prompts into a frozen pretrained model to adapt it to long-tailed data. For better …

Towards Understanding Why Mask Reconstruction Pretraining Helps In Downstream Tasks, Jiachun Pan, Pan Zhou, Shuicheng Yan

Research Collection School Of Computing and Information Systems

For unsupervised pretraining, mask-reconstruction pretraining (MRP) approaches, e.g. MAE (He et al., 2021) and data2vec (Baevski et al., 2022), randomly mask input patches and then reconstruct the pixels or semantic features of these masked patches via an auto-encoder. Then for a downstream task, supervised fine-tuning the pretrained encoder remarkably surpasses the conventional “supervised learning" (SL) trained from scratch. However, it is still unclear 1) how MRP performs semantic feature learning in the pretraining phase and 2) why it helps in downstream tasks. To solve these problems, we first theoretically show that on an auto-encoder of a two/one-layered convolution encoder/decoder, MRP …

Blind Fighter – A Video Game For The Visually Impaired, Avery Wayne Harrah

ATU Scholars Symposium

Blind Fighter is a video game made to be playable by anyone, regardless of any visual impairments the player may have. The game relies on auditory queues to allow players to understand what is happening in the game without ever having to see the screen. The project’s goal is to serve as a proof of concept that video games can be made inclusive with a few additions during development, without sacrificing overall quality. To do this, the game features full graphics in addition to testing many strategies for visually impaired players, including direction-based audio, unique sound effects for each game …

The Ozark Getaway, Houston Barber, Marcus Gasca, Evan Reece Matlock, Avery Wayne Harrah

ATU Scholars Symposium

Website designed for usage of renting AirBnb houses outside of the original website under the specific owner.

Creating And Evaluating Dimensional Analysis Software For University Students, Ac Hýbl

Campus Research Month

Most scientific disciplines depend on mathematics to varying degrees. Real-world problems often include quantifiable measurements with units. For example, chemistry, physics, and pharmacology require flawless unit conversions and dimensional homogeneity to obtain acceptable results. Students often choose to ignore units until the end of the problem-solving process, but this introduces errors and robs students of a deep understanding of units. Current tools for teaching dimensional analysis are limited both in scope and accessibility. Unit Playground addresses this issue by providing an interactive interface to experiment with units and their relationships.

A Study Of A Collaborative Task Management Application Built On React Native Using The Basic Ux Framework, Andrei Modiga

Campus Research Month

Many times it can be difficult to accomplish all this is proposed in a meeting. This project aimed to build a simple planner application using React Native that allows groups of people to collaborate and stay organized. The application was built using the BASIC Framework as a guide, and featured a collaboration feature that enabled users to share tasks, projects, and communicate with one another in order to stay coordinated and productive. The user interface was designed for easy use, allowing for quick and efficient task management within a group. The goal of the application was to provide a useful …

Visualizing Literary Narratives With A Graph-Centered Approach., Meg Ermer

Campus Research Month

The art of storytelling is multifaceted and nonlinear, involving multiple characters, themes, and symbols while often jumping between the present and past. While media forms such as novels can encapsulate these complexities, it is often difficult to visualize a narrative in an easy-to-understand format. Our contribution is a graph-based system to let users organize and visualize those narratives. Events and characters are represented as nodes and their relationships are represented as edges. Neo4J is used as a database management system to store the graph and to run queries on it, and Streamlit and Pyvis are used to represent the database …

Counterventions: A Reparative Reflection On Interventionist Hci, Rua Mae Williams, Louanne E. Boyd, Juan E. Gilbert

Engineering Faculty Articles and Research

Research in HCI applied to clinical interventions relies on normative assumptions about which bodies and minds are healthy, valuable, and desirable. To disrupt this normalizing drive in HCI, we define a “counterventional approach” to intervention technology design informed by critical scholarship and community perspectives. This approach is meant to unsettle normative assumptions of intervention as urgent, necessary, and curative. We begin with a historical overview of intervention in HCI and its critics. Then, through reparative readings of past HCI projects in autism intervention, we illustrate the emergent principles of a counterventional approach and how it may manifest research outcomes that …

Subgraph Centralization: A Necessary Step For Graph Anomaly Detection, Zhong Zhuang, Kai Ming Ting, Guansong Pang, Shuaibin Song

Research Collection School Of Computing and Information Systems

Abstract Graph anomaly detection has attracted a lot of interest recently. Despite their successes, existing detectors have at least two of the three weaknesses: (a) high computational cost which limits them to small-scale networks only; (b) existing treatment of subgraphs produces suboptimal detection accuracy; and (c) unable to provide an explanation as to why a node is anomalous, once it is identified. We identify that the root cause of these weaknesses is a lack of a proper treatment for subgraphs. A treatment called Subgraph Centralization for graph anomaly detection is proposed to address all the above weaknesses. Its importance is …

Open-Set Domain Adaptation By Deconfounding Domain Gaps, Xin Zhao, Shengsheng Wang, Qianru Sun

Research Collection School Of Computing and Information Systems

Open-Set Domain Adaptation (OSDA) aims to adapt the model trained on a source domain to the recognition tasks in a target domain while shielding any distractions caused by open-set classes, i.e., the classes “unknown” to the source model. Compared to standard DA, the key of OSDA lies in the separation between known and unknown classes. Existing OSDA methods often fail the separation because of overlooking the confounders (i.e., the domain gaps), which means their recognition of “unknown classes” is not because of class semantics but domain difference (e.g., styles and contexts). We address this issue by explicitly deconfounding domain gaps …

Stargazer: An Interactive Camera Robot For Capturing How-To Videos Based On Subtle Instructor Cues, Jiannan Li, Mauricio Sousa, Karthik Mahadevan, Bryan Wang, Paula Akemi Aoyagui, Nicole Yu, Angela Yang, Ravin Balakrishnan, Anthony Tang, Tovi Grossman

Research Collection School Of Computing and Information Systems

Live and pre-recorded video tutorials are an effective means for teaching physical skills such as cooking or prototyping electronics. A dedicated cameraperson following an instructor’s activities can improve production quality. However, instructors who do not have access to a cameraperson’s help often have to work within the constraints of static cameras. We present Stargazer, a novel approach for assisting with tutorial content creation with a camera robot that autonomously tracks regions of interest based on instructor actions to capture dynamic shots. Instructors can adjust the camera behaviors of Stargazer with subtle cues, including gestures and speech, allowing them to fluidly …

Para Cima Y Pa’ Abajo: Building Bridges Between Hci Research In Latin America And In The Global North, Pedro Reynolds-Cuéllar, Marisol Wong-Villacres, Karla A. Badillo-Urquiola, Mayra Donaji Barrera-Machuca, Franceli L. Cibrian, Marianela Ciolfi Felice, Carolina Fuentes, Laura Sanely Gaytán-Lugo, Vivian Genaro Motti, Monica Perusquía-Hernández, Oscar A. Lemus

Engineering Faculty Articles and Research

The Human-computer Interaction (HCI) community has the opportunity to foster the integration of research practices across the Global South and North to begin overcoming colonial relationships. In this paper, we focus on the case of Latin America (LATAM), where initiatives to increase the representation of HCI practitioners lack a consolidated understanding of the practices they employ, the factors that influence them, and the challenges that practitioners face. To address this knowledge gap, we employ a mixed-methods approach, comprising a survey (66 respondents) and in-depth interviews (19 interviewees). Our analyses characterize a set of research perspectives on how HCI is practiced …

Making Music Social: Creating A Spotify-Based Social Media Platform, Dalton J. Craven

Senior Theses

DKMS is a new type of social media platform for music lovers and groups of friends. It integrates tightly with Spotify, one of the largest music streaming services in the world. Users of DKMS can see what their friends are listening to, receive recommendations of new songs to listen to, and analyze their several key numerical metrics (happiness, danceability, loudness, and energy) of their top songs.

DKMS was built as part of the year-long Capstone senior design course at the University of South Carolina. A deployed app is visible at https://dkms.vercel.app, and the open-source code is visible at https://github.com/SCCapstone/DKMS.

Supporting Novices Author Audio Descriptions Via Automatic Feedback, Rosiana Natalie, Joshua Shi-Hao Tseng, Hernisa Kacorri, Kotaro Hara

Research Collection School Of Computing and Information Systems

Audio descriptions (AD) make videos accessible to those who cannot see them. But many videos lack AD and remain inaccessible as traditional approaches involve expensive professional production. We aim to lower production costs by involving novices in this process. We present an AD authoring system that supports novices to write scene descriptions (SD)—textual descriptions of video scenes—and convert them into AD via text-to-speech. The system combines video scene recognition and natural language processing to review novice-written SD and feeds back what to mention automatically. To assess the effectiveness of this automatic feedback in supporting novices, we recruited 60 participants to …

Robert X House Collection, Dan Bonenberger, Brooke Boyst, Franklin Haywood, Carrie Malas, Jaclyn Panter, Ian Tomashik, Laura Waskiwicz, Taylor Williams, Austin Martin

Digital Heritage Preservation Collection

This collection examines the Robert X House at 18827 Keystone Street in Detroit, Michigan, a significant site in the history of the Nation of Islam and the broader narrative of Black empowerment and civil rights in mid-20th century America. Originally built in 1951 as a modest Federal Housing Administration (FHA) minimum ranch-style home, the property was purchased by Robert Davenport (later known as Robert X), an influential leader in the Nation of Islam. The house became a hub for religious instruction, community-building, and activism.

Robert X, a former boxer and musician, became a prominent figure in Detroit’s Nation of Islam …

Object-Oriented Creative Coding For Digital Art Students, Dale E. Parson

Computer Science and Information Technology Faculty

An introductory course in creative graphical coding for both computer science and digital art majors need not be watered down. The availability of an excellent Java-based framework including an IDE, debugger, and high-level class and function library avoids potentially problematic mathematics and device-control demands by encapsulating them within the library. Students can add or modify a few lines of code and then run them from the IDE without explicit compilation steps or tool changes, immediately seeing the results of initial coding or bug fixes in the form of animated graphical objects. Incremental introduction of object-oriented mechanisms within the course such …