Graphics and Human Computer Interfaces Commons^™

Stylegan-∞: Extending Stylegan To Arbitrary-Ratio Translation With Stylebook, Yihua Dai, Tianyi Xiang, Bailin Deng, Yong Du, Hongmin Cai, Jing Qin, Shengfeng He

Research Collection School Of Computing and Information Systems

Although pre-trained large-scale generative models StyleGAN series have proven to be effective in various editing and translation tasks, they are limited to pre-defined fixed aspect ratio. To overcome this limitation, we propose StyleGAN-∞, a model that enables pre-trained StyleGAN to perform arbitrary-ratio conditional synthesis. Our key insight is to distill the expressive StyleGAN features into a StyleBook, such that an arbitrary-ratio condition can be translated to other forms by properly assembling pre-defined StyleBook vectors. To learn and leverage the StyleBook, we employ a network with three distinct stages, each corresponding to StyleBook extraction, StyleBook correspondence learning, and arbitrary-ratio synthesis. Extensive …

Map As A By-Product: Collective Landmark Mapping From Imu Data And User-Provided Texts In Situated Tasks, Ryo Yonetani, Kotaro Hara

Research Collection School Of Computing and Information Systems

This paper presents Collective Landmark Mapper, a novel map-as-a-by-product system for generating semantic landmark maps of indoor environments. Consider users engaged in situated tasks that require them to navigate these environments and regularly take notes on their smartphones. Collective Landmark Mapper exploits the smartphone's IMU data and the user's free text input during these tasks to identify a set of landmarks encountered by the user. The identified landmarks are then aggregated across multiple users to generate a unified map representing the positions and semantic information of all landmarks. In developing the proposed system, we focused specifically on retail applications and …

Unambiguous Granularity Distillation For Asymmetric Image Retrieval, Hongrui Zhang, Yi Xie, Haoquan Zhang, Cheng Xu, Xuandi Luo, Donglei Chen, Xuemiao Xu, Huaidong Zhang, Pheng Ann Heng, Shengfeng He

Research Collection School Of Computing and Information Systems

Previous asymmetric image retrieval methods based on knowledge distillation have primarily focused on aligning the global features of two networks to transfer global semantic information from the gallery network to the query network. However, these methods often fail to effectively transfer local semantic information, limiting the fine-grained alignment of feature representation spaces between the two networks. To overcome this limitation, we propose a novel approach called Layered-Granularity Localized Distillation (GranDist). GranDist constructs layered feature representations that balance the richness of contextual information with the granularity of local features. As we progress through the layers, the contextual information becomes more detailed, …

Hvi: A New Color Space For Low-Light Image Enhancement, Qingsen Yan, Yixu Feng, Cheng Zhang, Guansong Pang, Kangbiao Shi, Peng Wu, Wei Dong, Jinqiu Sun, Yanning Zhang

Research Collection School Of Computing and Information Systems

Low-Light Image Enhancement (LLIE) is a crucial computer vision task that aims to restore detailed visual information from corrupted low-light images. Many existing LLIE methods are based on standard RGB (sRGB) space, which often produce color bias and brightness artifacts due to inherent high color sensitivity in sRGB. While converting the images using Hue, Saturation and Value (HSV) color space helps resolve the brightness issue, it introduces significant red and black noise artifacts. To address this issue, we propose a new color space for LLIE, namely Horizontal/Vertical-Intensity (HVI), defined by polarized HS maps and learnable inten sity. The former enforces …

Programming A More Efficient Onboarding Process For New Employees, Long H. Pham

Undergraduate Honors Theses

The current onboarding process for new hires in the University of San Diego’s Shiley-Marcos School of Engineering is inefficient. There is no central location where new hires and administrators can track onboarding progress. Both parties have to manage multiple email chains and write their own reminders to keep track of everything. This leads to delays, missing deadlines, and confusion for both parties. A web-based onboarding application has been developed recently to address these issues and streamline the onboarding process for new hires. However, this application contains several accessibility issues and does not follow all of the standards for effective employee …

Bridging Cattle Farming And Technology: The Development Of Moomanager, Matthew Hayes

Honors College Theses

Small-scale cattle producers face persistent challenges in adopting digital tools for herd management, often due to barriers such as limited digital literacy, software complexity, and poor alignment with practical workflows. Existing literature highlights the potential benefits of mobile applications in agricultural contexts, yet adoption rates remain low among smaller operations. This thesis investigates how a streamlined, mobile-first application can address these adoption barriers while supporting essential farm management tasks. The study details the design and development of MooManager, a mobile application built with React Native and Supabase and structured around core features such as cattle tracking, beef sales logging, and …

Reducing Stigma Around Neurodiversity Through The Use Of Celebratory Technology Ice Breakers In First-Year Undergraduate Classrooms, Briana Craig

Electrical Engineering and Computer Science (MS) Theses

Celebratory technology for Neurodiversity is a new paradigm in the field of human computer interaction; it focuses on reducing stigma surrounding neurodivergent labels and behaviors. Celebratory technology aims to highlight the strengths of neurodiversity rather than fixing socially undesired traits, shifting the responsibility for change from neurodivergent individuals to society's attitudes. Stigma reduction can be accomplished through providing high quality interactions, where anyone can meet and learn about positive traits in others as well as learn of interests' others have in common, thus reframing neurodivergence as inclusion in human diversity rather than a condition to be stigmatized or objectified. This …

Improving Home Security Through User Centered Device Positioning, Lalith Nadipalli

Theses and Dissertations

In today’s world, where technology is advancing rapidly and security threats are becoming more complex, the need for effective home safety measures is more critical than ever. Homeowners are increasingly turning to a variety of smart devices, such as smoke detectors, carbon monoxide detectors, and security cameras, to protect their living spaces against potential dangers like burglary, fire, and environmental hazards. These devices offer essential protection, acting as both early warning systems and visual surveillance tools. However, their effectiveness largely hinges on how well they are placed within the home. Proper placement of these safety devices ensures that they provide …

Ml Playground: Data Modification/Preprocessing And Model Simulation Tool, Marco D. Cerrato

Electronic Theses, Projects, and Dissertations

There is a heavy reliance on programming when it comes to learning machine learning (ML). This often creates barriers for students and newcomers unfamiliar with coding. While the lessons you learn in the classroom provide essential foundational understanding, some technical or practical aspects of ML—such as data preprocessing, feature engineering, and model tuning—are best learned through hands-on interaction. ML Playground was developed to act as a proof-of-concept application to address this gap by offering a browser-based, graphical user interface that lets users engage with core ML workflows without writing code. Designed with educational accessibility in mind, the application allows users …

“I Can Run At Night!”: Using Augmented Reality To Support Nighttime Guided Running For Low-Vision Runners, Yuki Abe, Keisuke Matsushima, Kotaro Hara, Daisuke Sakamoto, Tetsuo Ono

Research Collection School Of Computing and Information Systems

Dark environment challenges low-vision (LV) individuals to engage in running by following sighted guide—a Caller-style guided running—due to insufficient illumination, because it prevents them from using their residual vision to follow the guide and be aware about their environment. We design, develop, and evaluate RunSight, an augmented reality (AR)-based assistive tool to support LV individuals to run at night. RunSight combines see-through HMD and image processing to enhance one’s visual awareness of the surrounding environment (e.g., potential hazard) and visualize the guide’s position with AR-based visualization. To demonstrate RunSight’s efficacy, we conducted a user study with 8 LV runners. The …

Simplifying 3d Printing Using Natural Language Processing, Jared E. Rosenberger

Undergraduate Theses

3D printing is a crucial technology with many applications in different fields. To be able to use this technology to its full extent, expertise in computer aided design (CAD) technology and 3D modeling is required. Many people interested in 3D printing do not have this expertise and thus cannot build custom models, and are consequently forced to buy them instead. Natural language processing (NLP) is one tool that can vastly simplify 3D modeling for those lacking CAD experience. Using NLP, someone can simply dictate what they want to be able to print, and a computer can then build a 3D …

One-For-All: Towards Universal Domain Translation With A Single Stylegan, Yong Du, Jiahui Zhan, Xinzhe Li, Junyu Dong, Sheng Chen, Ming-Hsuan Yang, Shengfeng He

Research Collection School Of Computing and Information Systems

In this paper, we propose a novel translation model, UniTranslator, for transforming representations between visually distinct domains under conditions of limited training data and significant visual differences. The main idea behind our approach is leveraging the domain-neutral capabilities of CLIP as a bridging mechanism, while utilizing a separate module to extract abstract, domain-agnostic semantics from the embeddings of both the source and target realms. Fusing these abstract semantics with target-specific semantics results in a transformed embedding within the CLIP space. To bridge the gap between the disparate worlds of CLIP and StyleGAN, we introduce a new non-linear mapper, the CLIP2P …

Neurovig: Integrating Event Cameras For Resource-Efficient Video Grounding, Dulanga Weerakoon, Vigneshwaran Subbaraju, Joo Hwee Lim, Archan Misra

Research Collection School Of Computing and Information Systems

Spatio-Temporal Video Grounding (STVG) - the task of identifying the target object in the field-of-view that the language instruction refers to - is a fundamental vision-language task. Current STVG approaches typically utilize feeds from an RGB camera that is assumed to be always-on and process the video frames using complex neural network pipelines. As a result they often impose prohibitive system overheads (energy latency) on pervasive devices. To address this we propose NeuroViG with two key innovations: (a) leveraging on event streams from a low-power neuromorphic event camera sensor to perform selective triggering of the more energy-hungry RGB camera for …

Learning An Interpretable Stylized Subspace For 3d-Aware Animatable Artforms, Chenxi Zheng, Bangzhen Liu, Xuemiao Xu, Huaidong Zhang, Shengfeng He

Research Collection School Of Computing and Information Systems

Throughout history, static paintings have captivated viewers within display frames, yet the possibility of making these masterpieces vividly interactive remains intriguing. This research paper introduces 3DArtmator, a novel approach that aims to represent artforms in a highly interpretable stylized space, enabling 3D-aware animatable reconstruction and editing. Our rationale is to transfer the interpretability and 3D controllability of the latent space in a 3D-aware GAN to a stylized sub-space of a customized GAN, revitalizing the original artforms. To this end, the proposed two-stage optimization framework of 3DArtmator begins with discovering an anchor in the original latent space that accurately mimics the …

Density Boosts Everything: A One-Stop Strategy For Improving Performance, Robustness, And Sustainability Of Malware Detectors, Jianwen Tian, Wei Kong, Debin Gao, Tong Wang, Taotao Gu, Kefan Qiu, Zhi Wang, Xiaohui Kuang

Research Collection School Of Computing and Information Systems

In the contemporary landscape of cybersecurity, AI-driven detectors have emerged as pivotal in the realm of malware detection. However, existing AI-driven detectors encounter a myriad of challenges, including poisoning attacks, evasion attacks, and concept drift, which stem from the inherent characteristics of AI methodologies. While numerous solutions have been proposed to address these issues, they often concentrate on isolated problems, neglecting the broader implications for other facets of malware detection. This paper diverges from the conventional approach by not targeting a singular issue but instead identifying one of the fundamental causes of these challenges, sparsity. Sparsity refers to a scenario …

Heartdj - Music Recommendation And Generation Through Biofeedback From Heart Rate Variability, Egemen Şahin

Dartmouth College Master’s Theses

This study investigates the integration of real-time physiological data with AI-generated music to enhance emotional well-being, stress regulation, and focus, using Heart Rate Variability (HRV) as a biomarker of autonomic function. Conducted in two phases—Stable Audio Open (SAO) and Suno (SUNO)—the research evaluates biofeedback-driven music interventions across varying daily music-listening habits.

In the SAO phase, short AI-generated instrumental tracks were compared with Spotify recommendations and guided meditation. Modest HRV improvements were observed in biofeedback conditions, but participants noted emotional limitations, citing short track lengths and abrupt transitions.

The SUNO phase addressed these limitations with longer, more complex AI-generated compositions combined …

Dashar: An Implementation Of Augmented Reality Technology For Automotive Applications, Trevor D. Brown

Masters Theses & Specialist Projects

Since the advent of the modern automobile, manufacturers have provided means of tracking various critical data points associated with automobile operation, with the most prominent and standardized method being the instrument cluster. These data points include, but are not limited to, automobile speed, engine speed, fuel level, oil temperature, radiator (water) temperature, and battery charge. While this data is updated in real-time as the automobile is running, traditional instrument clusters cannot be modified or adjusted to the automobile driver’s needs, unless extensive after-market modifications are made. These modifications can be expensive, and require great understanding of the automobile’s assembly.

Alongside …

Openmuse: Integrating Open-Source Models Into Music Creation Workflows, Tyler K. Vergho

Dartmouth College Master’s Theses

This master's thesis introduces OpenMUSE (Open Multimodal Unified Sound Engine), a platform that demonstrates the potential of open-source AI music generation by integrating state-of-the-art deep learning models into a unified system. By unifying ten different open-source models, including MusicGen, AudioLDM2, and custom-trained text-to-symbolic music generation models, OpenMUSE aims to create a user-friendly interface that empowers artists to produce complex, adaptive musical compositions. The system enhances accessibility by providing a simple web interface and natural language controls, while improving controllability through features like melody conditioning and semantic audio editing. Specifically, OpenMUSE offers a digital audio workstation (DAW)-inspired interface that lowers the …

Procedural Terrain Generation: Noise Functions, Modern Methods, And Style Transfer, Hunter Barton

EWU Masters Thesis Collection

Procedural terrain generation, the algorithmic creation of digital terrain, finds use in multiple types of digital media. As the capabilities of modern computation increase, the ability to create more and more realistic terrains fully procedurally at scale improves. Modern methods of procedural generation have also overlapped with these advances, most notably advances in hardware. To account for this, a survey was done of modern methods for procedural terrain generation. Smooth procedural noise functions are one of the backbones of procedural terrain generation. Perlin noise, value noise, and fractal noise were explored in-depth. These noise functions were also tested for capabilities …

Automatic Scoring Cornhole Board System, Eric Diffendal, Jonah Harsh, Connor Lengel, Brett Sukie

Williams Honors College, Honors Research Projects

The objective is to create a self-scoring cornhole board that can detect and calculate each team's score based on the bags thrown each round and to be created at a low cost/eventually being sold at the current cost of a normal board. When playing cornhole, the game is simple: throw a bag on the board; however, the scores are variable (deduct and add) across each round. The most common issue when playing cornhole is miscalculations of the scores and forgetting the correct scores. Thus, this invention will make gameplay easy for all to play.

Visualization Of Paleocurrents On A Web Application Using Gplates, Anjan Sapkota

MS in Computer Science Theses

Paleocurrents are flow directions derived from features of sedimentary rocks that reveal the direction of the current of wind or water that deposited the sediment. In 2015, Brand et al. created a global database of paleocurrents, which contains over 1,000,000 measurements worldwide: North America, South America, Australia, Great Britain, parts of Western Europe, China, Africa are fairly well represented; Antarctica, Eastern Europe, and Asia are modestly represented and Russia is poorly represented. The contribution of this thesis is a web application that uses the GPlates’ Application Programming Interface (API) to visualize global paleocurrents through time in an interactive way based …

Triadic Temporal-Semantic Alignment For Weakly-Supervised Video Moment Retrieval, Jin Liu, Jialong Xie, Fengyu Zhou, Shengfeng He

Research Collection School Of Computing and Information Systems

Video Moment Retrieval (VMR) aims to identify specific event moments within untrimmed videos based on natural language queries. Existing VMR methods have been criticized for relying heavily on moment annotation bias rather than true multi-modal alignment reasoning. Weakly supervised VMR approaches inherently overcome this issue by training without precise temporal location information. However, they struggle with fine-grained semantic alignment and often yield multiple speculative predictions with prolonged video spans. In this paper, we take a step forward in the context of weakly supervised VMR by proposing a triadic temporalsemantic alignment model. Our proposed approach augments weak supervision by comprehensively addressing …

Digital Twin For Shelf Intelligence: Ai-Driven Inventory Management For Minimizing Food Waste, Charlotte Maples, Marvin Velazquez

College of Engineering Summer Undergraduate Research Program

This project aims to develop a solution for improving grocery store inventory management by leveraging AI-driven image recognition. Traditional inventory methods, which rely on manual counting or barcode scanning, are inefficient, labor-intensive, and prone to human error. Over an 8-week period, we designed and developed a basic iPad app capable of identifying specific types of fruit and automatically updating inventory records in real time. By utilizing the iPad’s camera and machine learning algorithms, the app demonstrates the potential to streamline inventory tracking, reduce manual labor, and improve accuracy in managing perishable goods. Future work will focus on expanding the app’s …

Enhancing Place-Based Interaction With Emotion Ai And Augmented Reality, Jake Maier, Ivan Martinez

College of Engineering Summer Undergraduate Research Program

This project explores the integration of augmented reality (AR) and Emotion AI technologies to enhance user experiences in physical environments. By seamlessly merging virtual elements with real-world contexts, we aim to deepen individuals’ interactions and perceptions of their surroundings. Leveraging AR technology enables users to access contextual information, engage with interactive content, and navigate spaces with heightened immersion and understanding. Additionally, Emotion AI enhances these experiences by detecting and responding to users’ emotional states, fostering personalized and emotionally resonant interactions. We aim to integrate digital content within physical environments using mixed-reality headsets equipped with eye-tracking capabilities and consumer-grade wireless EEG …

Granular3d: Delving Into Multi-Granularity 3d Scene Graph Prediction, Kaixiang Huang, Jingru Yang, Jin Wang, Shengfeng He, Zhan Wang, Haiyan He, Qifeng Zhang, Guodong Lu

Research Collection School Of Computing and Information Systems

This paper addresses the significant challenges in 3D Semantic Scene Graph (3DSSG) prediction, essential for understanding complex 3D environments. Traditional approaches, primarily using PointNet and Graph Convolutional Networks, struggle with effectively extracting multi-grained features from intricate 3D scenes, largely due to a focus on global scene processing and single-scale feature extraction. To overcome these limitations, we introduce Granular3D, a novel approach that shifts the focus towards multi-granularity analysis by predicting relation triplets from specific sub-scenes. One key is the Adaptive Instance Enveloping Method (AIEM), which establishes an approximate envelope structure around irregular instances, providing shape-adaptive local point cloud sampling, thereby …

Certified Robust Accuracy Of Neural Networks Are Bounded Due To Bayes Errors, Ruihan Zhang, Jun Sun

Research Collection School Of Computing and Information Systems

Adversarial examples pose a security threat to many critical systems built on neural networks. While certified training improves robustness, it also decreases accuracy noticeably. Despite various proposals for addressing this issue, the significant accuracy drop remains. More importantly, it is not clear whether there is a certain fundamental limit on achieving robustness whilst maintaining accuracy. In this work, we offer a novel perspective based on Bayes errors. By adopting Bayes error to robustness analysis, we investigate the limit of certified robust accuracy, taking into account data distribution uncertainties. We first show that the accuracy inevitably decreases in the pursuit of …

Hierarchical Damage Correlations For Old Photo Restoration, Weiwei Cai, Xuemiao Xu, Jiajia Xu, Huaidong Zhang, Haoxin Yang, Kun Zhang, Shengfeng He

Research Collection School Of Computing and Information Systems

Restoring old photographs can preserve cherished memories. Previous methods handled diverse damages within the same network structure, which proved impractical. In addition, these methods cannot exploit correlations among artifacts, especially in scratches versus patch-misses issues. Hence, a tailored network is particularly crucial. In light of this, we propose a unified framework consisting of two key components: ScratchNet and PatchNet. In detail, ScratchNet employs the parallel Multi-scale Partial Convolution Module to effectively repair scratches, learning from multi-scale local receptive fields. In contrast, the patch-misses necessitate the network to emphasize global information. To this end, we incorporate a transformer-based encoder and decoder …

Crime Prediction Using Agent-Based Modeling, Yifei Gong

Dissertations, Theses, and Capstone Projects

Crime risk evaluation and crime prediction using agent-based modeling (ABM) have gained popularity in the field of computational criminology in recent years. Traditionally, researchers rely on statistical methods and machine learning models to predict crimes using historical data. ABM generates macro-level crime patterns in a bottom-up fashion by simulating the daily behaviors of autonomous entities, such as citizens and offenders. ABM takes into consideration the non-linear interactions between agents under complex social contexts. Currently, the comprehensive usage of ABM for criminological theory testing and urban policy evaluations calls for a unified software framework. In this research, we introduce CARESim, an …

Vaid: Indexing View Designs In Visual Analytics System, Lu Ying, Aoyu Wu, Haotian Li, Zikun Deng, Ji Lan, Jiang Wu, Yong Wang, Huamin Qu, Dazhen Deng, Yingcai Wu

Research Collection School Of Computing and Information Systems

Visual analytics (VA) systems have been widely used in various application domains. However, VA systems are complex in design, which imposes a serious problem: although the academic community constantly designs and implements new designs, the designs are difficult to query, understand, and refer to by subsequent designers. To mark a major step forward in tackling this problem, we index VA designs in an expressive and accessible way, transforming the designs into a structured format. We first conducted a workshop study with VA designers to learn user requirements for understanding and retrieving professional designs in VA systems. Thereafter, we came up …

Instant3d: Instant Text-To-3d Generation, Ming Li, Pan Zhou, Jia-Wei Liu, Jussi Keppo, Shuicheng Yan, Xiangyu Xu

Research Collection School Of Computing and Information Systems

Text-to-3D generation has attracted much attention from the computer vision community. Existing methods mainly optimize a neural field from scratch for each text prompt, relying on heavy and repetitive training cost which impedes their practical deployment. In this paper, we propose a novel framework for fast text-to-3D generation, dubbed Instant3D. Once trained, Instant3D is able to create a 3D object for an unseen text prompt in less than one second with a single run of a feedforward network. We achieve this remarkable speed by devising a new network that directly constructs a 3D triplane from a text prompt. The core …