Graphics and Human Computer Interfaces Commons^™

Real-Time Motion Augmentation And Synthesis For Animating The Hands And Eyes Of Virtual Humans And Avatars, Ryan Canales

All Dissertations

Virtual Reality (VR) enables users to interact within virtual worlds via an embodied virtual representation of themselves called an “avatar”. Because avatars are essential for immersive experiences, it is important to consider how altering or augmenting avatar motion affects virtual experiences. This dissertation aims to improve virtual experiences by addressing some of the many challenges in animating avatars and virtual humans.

In our first study, we addressed the lack of tactile feedback during virtual grasping by using visual feedback techniques. We augmented the avatar’s hand motion to remain outside virtual objects (“outer hand”) even when the user’s hand penetrated them. …

Habit Coach: Customising Rag-Based Chatbots To Support Behavior Change, Arian Fooroogh Mand Arabi, Cansu Koyuturk, Michael O'Mahony, Raffaella Calati, Dimitri Ognibene

Conference papers

This paper presents the iterative development of Habit Coach, a GPT-based chatbot designed to support users in habit change through personalized interaction. Employing a user-centered design approach, we developed the chatbot using a Retrieval-Augmented Generation (RAG) system, which enables behavior personalization without retraining the underlying language model (GPT-4). The system leverages document retrieval and specialized prompts to tailor interactions, drawing from Cognitive Behavioral Therapy (CBT) and narrative therapy techniques. A key challenge in the development process was the difficulty of translating declarative knowledge into effective interaction behaviors. In the initial phase, the chatbot was provided with declarative knowledge about CBT …

Smartphone Haptics Can Uncover Differences In Touch Interactions Between Asd And Neurotypicals, Ivonne Monarca, Franceli L. Cibrian, Isabel López Hurtado, Monica Tentori

Engineering Faculty Articles and Research

Utilizing touch interactions from smartphones for gathering data and identifying digital markers for screening and monitoring neurological disorders, such as Autism Spectrum Disorder (ASD), is an emerging area of research. Smartphones provide multiple benefits for this kind of study, including unobtrusive data collection via built-in sensors, integrated haptic feedback systems, and the capability to create specialized applications. Acknowledging the significant yet understudied presence of tactile processing differences in individuals with ASD, we designed and developed Feel and Touch, a mobile game that leverages the haptic capabilities of smartphones. This game provides vibrotactile feedback in response to touch interactions and collects …

Rubrics Informed By The Cognitive Theory Of Multimedia Learning That Support Research On Personalized Learning Paths, Sean A. Mochocki, Mark G. Reith, Jonathan Zemmer

AFIT Documents

Personalized Learning Paths (PLP)s are a popular area of research in E-Learning where sequences of Learning Materials (LM)s and activities are returned based on a learner profile, the LM metadata, and a knowledge structure that describes the relationship between the underlying topics. Unfortunately, PLP researchers tend to not use an empirically supported cognitive science framework for their research, instead relying on such unsupported theories as learning styles or developing their own ad hoc approaches. While many of these researchers present and solve challenging PLP problems using a variety of algorithmic approaches, the PLP community in general would benefit from a …

Unlocking Potential: Analyzing The Content, Style, Structure, And Interactivity Of Mesonets As Operational Dashboards, Savannah Olivas, Jeannette Sutton, Michele K. Olson

Emergency Preparedness, Homeland Security, and Cybersecurity Faculty Scholarship

Emergency managers need data and information to make life-saving decisions on behalf of the public. Operational dashboards, if designed appropriately, can provide this information in a central location and reduce cognitive demands during decision-making. Mesonet websites can serve as a type of operational dashboard that has the potential to provide the meteorological data necessary for emergency managers to make decisions. In this study, we use quantitative content analysis to examine the content, style, structure, and interactivity of 18 Mesonet websites from across the contiguous United States. We find that Mesonet websites vary in the type and amount of content they …

Eyetraes : Fine-Grained, Low-Latency Eye Tracking Via Adaptive Event Slicing, Argha Sen, Panahetipola Mudiyanselage Nuwan Bandara, Ila Gokarn, Thivya Kandappu, Archan Misra

Research Collection School Of Computing and Information Systems

Eye-tracking technology has gained significant attention in recent years due to its wide range of applications in humancomputer interaction, virtual and augmented reality, and wearable health. Traditional RGB camera-based eye-tracking systems often struggle with poor temporal resolution and computational constraints, limiting their effectiveness in capturing rapid eye movements. To address these limitations, we propose EyeTrAES, a novel approach using neuromorphic event cameras for high-fidelity tracking of natural pupillary movement that shows significant kinematic variance. One of EyeTrAES’s highlights is the use of a novel adaptive windowing/slicing algorithm that ensures just the right amount of descriptive asynchronous event data accumulation within …

Hi3d: Pursuing High-Resolution Image-To-3d Generation With Video Diffusion Models, Haibo Yang, Yang Chen, Yingwei Pan, Ting Yao, Zhineng Chen, Chong-Wah Ngo, Tao Mei

Research Collection School Of Computing and Information Systems

Despite having tremendous progress in image-to-3D generation, existing methods still struggle to produce multi-view consistent images with high-resolution textures in detail, especially in the paradigm of 2D diffusion that lacks 3D awareness. In this work, we present High-resolution Image-to-3D model (Hi3D), a new video diffusion based paradigm that redefines a single image to multi-view images as 3D-aware sequential image generation (i.e., orbital video generation). This methodology delves into the underlying temporal consistency knowledge in video diffusion model that generalizes well to geometry consistency across multiple views in 3D generation. Technically, Hi3D first empowers the pre-trained video diffusion model with 3D-aware …

Eyegraph : Modularity-Aware Spatio Temporal Graph Clustering For Continuous Event-Based Eye Tracking, Panahetipola Mudiyanselage Nuwan Bandara, Thivya Kandappu, Archan Misra, Ila Gokarn, Archan Misra

Research Collection School Of Computing and Information Systems

Continuous tracking of eye movement dynamics plays a significant role in developing a broad spectrum of human-centered applications, such as cognitive skills (visual attention and working memory) modeling, human-machine interaction, biometric user authentication, and foveated rendering. Recently neuromorphic cameras have garnered significant interest in the eye-tracking research community, owing to their sub-microsecond latency in capturing intensity changes resulting from eye movements. Nevertheless, the existing approaches for event-based eye tracking suffer from several limitations: dependence on RGB frames, label sparsity, and training on datasets collected in controlled lab environments that do not adequately reflect real-world scenarios. To address these limitations, in …

Cirp: Cross‑Item Relational Pre‑Training For Multimodal Product Bundling, Yunshan Ma, Yingzhi He, Wenjun Zhong, Xiang Wang, Roger Zimmermann, Tat-Seng Chua

Research Collection School Of Computing and Information Systems

Product bundling has been a prevailing marketing strategy that is beneficial in the online shopping scenario. Effective product bundling methods depend on high-quality item representations capturing both the individual items' semantics and cross-item relations. However, previous item representation learning methods, either feature fusion or graph learning, suffer from inadequate cross-modal alignment and struggle to capture the cross-item relations for cold-start items. Multimodal pre-train models could be the potential solutions given their promising performance on various multimodal downstream tasks. However, the cross-item relations have been under-explored in the current multimodal pre-train models.To bridge this gap, we propose a novel and simple …

Transitioning Our Website To Libguides Cms, Samantha Duncan, Eric Resnis

Library Faculty Presentations

In this presentation, we describe how we used data from rapid and in-depth student usability testing to assist with the redesign of the library’s website as we finally transitioned to LibGuides CMS. Using the Springy tools; LibGuides, LibGuides CMS, LibCal, and LibWizard we outlined how we were able to create and carry out this highly effective testing, resulting in a better understanding of how our students navigate our site and how to improve it. During this journey, attendees were provided with the detailed and some might say lengthy process that was undertaken to achieve our goals. We did this by …

The Psychological Impacts Of Algorithmic And Ai-Driven Social Media On Teenagers: A Call To Action, Sunil Arora, Sahil Arora, John Hastings

Research & Publications

This study investigates the meta-issues surrounding social media, which, while theoretically designed to enhance social interactions and improve our social lives by facilitating the sharing of personal experiences and life events, often results in adverse psychological impacts. Our investigation reveals a paradoxical outcome: rather than fostering closer relationships and improving social lives, the algorithms and structures that underlie social media platforms inadvertently contribute to a profound psychological impact on individuals, influencing them in unforeseen ways. This phenomenon is particularly pronounced among teenagers, who are disproportionately affected by curated online personas, peer pressure to present a perfect digital image, and the …

Construction And Application Of Multi-Dimensional Portrait System For Green Technology Innovation Enterprises In China: Taking The Green Transportation Technology Field As An Example, Wenke Hao, Jianlin Yang, Lei Miao

Journal of Scientific Information Research

[Purpose/significance]By constructing and applying the multi-dimensional portrait system of green technology innovation enterprises in China, this paper aims to comprehensively understand the status quo, advantages and obstacles of enterprises in specific fields in green technology innovation, so as to provide scientific references and suggestions for relevant government departments and decision makers of enterprises. [Method/process]Based on resource based view and environmental dependence theory, we select the internal and external labels of enterprises, and designs a multi-dimensional label system to objectively describe the performance of enterprises in terms of profitability,scientific research and innovation, public opinion and environmental responsibility. Then, the green technology …

Fast And High-Resolution View Synthesis From A Single Input Panorama, Nam Nguyen, Angela V. Chen, Theresa Zhu, Seth Johnson, Pranav Dumpa, Benjamin Geil

College of Engineering Summer Undergraduate Research Program

We introduce a novel method to convert a single input panorama into a 3D colored mesh representation of the scene. Unlike recent methods based on neural rendering, which are limited to low-resolution inputs and offline rendering, our approach supports 4k resolution inputs and real time rendering in a virtual reality headset. We first estimate a depth map and produce an initial layered depth image (LDI) representation. We fill unseen regions behind objects by iteratively cutting and inpainting the LDI. We then convert the LDI into an optimized, texture mapped mesh to achieve a compact representation

Digital Twin For Shelf Intelligence: Ai-Driven Inventory Management For Minimizing Food Waste, Charlotte Maples, Marvin Velazquez

College of Engineering Summer Undergraduate Research Program

This project aims to develop a solution for improving grocery store inventory management by leveraging AI-driven image recognition. Traditional inventory methods, which rely on manual counting or barcode scanning, are inefficient, labor-intensive, and prone to human error. Over an 8-week period, we designed and developed a basic iPad app capable of identifying specific types of fruit and automatically updating inventory records in real time. By utilizing the iPad’s camera and machine learning algorithms, the app demonstrates the potential to streamline inventory tracking, reduce manual labor, and improve accuracy in managing perishable goods. Future work will focus on expanding the app’s …

Enhancing Place-Based Interaction With Emotion Ai And Augmented Reality, Jake Maier, Ivan Martinez

College of Engineering Summer Undergraduate Research Program

This project explores the integration of augmented reality (AR) and Emotion AI technologies to enhance user experiences in physical environments. By seamlessly merging virtual elements with real-world contexts, we aim to deepen individuals’ interactions and perceptions of their surroundings. Leveraging AR technology enables users to access contextual information, engage with interactive content, and navigate spaces with heightened immersion and understanding. Additionally, Emotion AI enhances these experiences by detecting and responding to users’ emotional states, fostering personalized and emotionally resonant interactions. We aim to integrate digital content within physical environments using mixed-reality headsets equipped with eye-tracking capabilities and consumer-grade wireless EEG …

Leveraging Tradespace-Exploration For A Senior Project Team Formation Application, Miguel Saenz

College of Engineering Summer Undergraduate Research Program

This project revolves around the development of an app in MATLAB that leverages the VASSAR rule-based system and a genetic algorithm to form groups of teams for the Mechanical Engineering Senior Design project class. We leveraged the iterative design process to eventually attain a functional app with a reasonable runtime that works provided correctly formatted rulesheets describing student project preference and member preference.

An Efficient Fourier Caching Algorithm For Walk On Spheres, Zihong Zhou

Dartmouth College Master’s Theses

Walk on Spheres (WoS) is a grid-free Monte Carlo method for solving elliptic partial differential equations (PDEs).
Rather than discretizing the domain, WoS leverages the mean-value principle to obtain Monte Carlo estimates by recursively averaging the solution over the largest contained sphere, terminating upon reaching the boundary.
Unfortunately, WoS requires many independent estimates to achieve noise-free results.

We propose an acceleration technique for WoS, inspired by irradiance caching methods, that computes the solution at a sparse set of locations, and extrapolates these cached values to local neighborhoods. A key insight is that WoS can be extended to compute not only …

Enhancing Recipe Retrieval With Foundation Models: A Data Augmentation Perspective, Fangzhou Song, Bin Zhu, Yanbin Hao, Shuo Wang

Research Collection School Of Computing and Information Systems

Learning recipe and food image representation in common embedding space is non-trivial but crucial for cross-modal recipe retrieval. In this paper, we propose a new perspective for this problem by utilizing foundation models for data augmentation. Leveraging on the remarkable capabilities of foundation models (i.e., Llama2 and SAM), we propose to augment recipe and food image by extracting alignable information related to the counterpart. Specifically, Llama2 is employed to generate a textual description from the recipe, aiming to capture the visual cues of a food image, and SAM is used to produce image segments that correspond to key ingredients in …

Improving Out-Of-Distribution Detection With Disentangled Foreground And Background Features, Choubo Ding, Guansong Pang

Research Collection School Of Computing and Information Systems

Detecting out-of-distribution (OOD) inputs is a principal task for ensuring the safety of deploying deep-neural-network classifiers in open-set scenarios. OOD samples can be drawn from arbitrary distributions and exhibit deviations from in-distribution (ID) data in various dimensions, such as foreground features (e.g., objects in CIFAR100 images vs. those in CIFAR10 images) and background features (e.g., textural images vs. objects in CIFAR10). Existing methods can confound foreground and background features in training, failing to utilize the background features for OOD detection. This paper considers the importance of feature disentanglement in out-of-distribution detection and proposes the simultaneous exploitation of both foreground and …

Densetrack : Drone-Based Crowd Tracking Via Density-Aware Motion-Appearance Synergy, Yi Lei, Huilin Zhu, Jingling Yuan, Guangli Xiang, Xian Zhong, Shengfeng He

Research Collection School Of Computing and Information Systems

Drone-based crowd tracking faces difficulties in accurately identifying and monitoring objects from an aerial perspective, largely due to their small size and close proximity to each other, which complicates both localization and tracking. To address these challenges, we present the Density-aware Tracking (DenseTrack) framework. DenseTrack capitalizes on crowd counting to precisely determine object locations, blending visual and motion cues to improve the tracking of small-scale objects. It specifically addresses the problem of cross-frame motion to enhance tracking accuracy and dependability. DenseTrack employs crowd density estimates as anchors for exact object localization within video frames. These estimates are merged with motion …