Graphics and Human Computer Interfaces Commons^™

Towards Dynamic Context Detection From Voice Commands And Conversations With Smart Assistants In Smart Homes, Jeniya Sultana

Graduate Theses/Dissertations

Voice-enabled interactions have become increasingly popular with the rise of voice assistants. Identifying contexts or meanings from voice commands and conversations with smart assistants can contribute to the autonomous control of smart home devices and appliances. To improve automation, there is a growing need for efficient context detection that eliminates the need to memorize voice commands. To address this need, I followed a two-step approach in my research. In the first step, I developed a unique context recognition model using a transformer, an attention mechanism, and a fully connected neural network. I trained this model on a conversational dataset of …

Scene Understanding And Spatial Analysis Using Scene Graph Enhanced By Hall's Proxemics Zones In Smart Homes, Debaleen Das Spandan

Graduate Theses/Dissertations

Voice-controlled smart assistants have received widespread popularity. It plays a pivotal role in smart homes by providing a natural and convenient interface for interacting with smart devices. However, these assistants are unable to serve persons with physical disabilities and speech impairments. Therefore, non-verbal communication methods, such as eye tracking, gesture recognition, and context awareness can complement and overcome some of these limitations to enhance user experience in smart homes. To address this issue, I am investigating non-verbal communication methods to make smart home technology more accessible and intuitive. In this research, I focus on proxemics, i.e., the study of distance …

Flexible Attenuation Fields: Tomographic Reconstruction From Heterogeneous Datasets, Clifford S. Parker

Theses and Dissertations--Computer Science

Traditional reconstruction methods for X-ray computed tomography (CT) are highly constrained in the variety of input datasets they admit. Many of the imaging settings -- the incident energy, field-of-view, effective resolution -- remain fixed across projection images, and the only real variance is in the detector's position and orientation with respect to the scene. In contrast, methods for 3D reconstruction of natural scenes are extremely flexible to the geometric and photometric properties of the input datasets, readily accepting and benefiting from images captured under varying lighting conditions, with different cameras, and at disparate points in time and space. Extending CT …

Escape The Planet: Revolutionizing Game Design With Novel Oop Techniques, Qusai Kamal Fannoun

All Graduate Theses, Dissertations, and Other Capstone Projects

Mobile devices are continuously evolving and greater computing power and graphics capabilities are being introduced every year. As a result, there is an increasing demand for challenging and engaging mobile games that leverage these advanced features. This project explores best design practices using the development of Escape the Planet, which is an intricate maze game for mobile devices in which players navigate using a spaceship that is trapped in a hostile planet’s maze while avoiding obstacles and enemy attacks. The goal is to safely guide the spaceship out of the maze without colliding into walls or taking bullets from defensive …

Consistent Monte Carlo Methods For Non-Linear Applications In Light Transport, Zackary T. Misso

Dartmouth College Ph.D Dissertations

The study of light transport focuses on describing the propagation of light from emitters to sensors through accurately describing the interactions light can undergo with everything in between. Physically-based rendering is the process of applying the laws of light transport to formulate practical algorithms which simulate the flow of light for the purpose of synthesizing images of virtual environments.

Unfortunately, there are very few interesting scene configurations which can be computed analytically. Instead, modern solutions predominantly rely on Monte Carlo integration to stochastically estimate the transfer of light since the process is both unbiased and consistent. Meaning, it is expected …

Predicting An Optimal Medication/Prescription Regimen For Patient Discordant Chronic Comorbidities Using Multi-Output Models, Ichchha Pradeep Sharma, Tam Nguyen, Shruti Ajay Singh, Tom Ongwere

Computer Science Faculty Publications

This paper focuses on addressing the complex healthcare needs of patients struggling with discordant chronic comorbidities (DCCs). Managing these patients within the current healthcare system often proves to be a challenging process, characterized by evolving treatment needs necessitating multiple medical appointments and coordination among different clinical specialists. This makes it difficult for both patients and healthcare providers to set and prioritize medications and understand potential drug interactions. The primary motivation of this research is the need to reduce medication conflict and optimize medication regimens for individuals with DCCs. To achieve this, we allowed patients to specify their health conditions and …

The Process Of Video Game Development: Watermelon Willy, Jade Westgor

University Honors Program Senior Projects

Developing a video game requires and provides skills that can be very helpful for computer science students, as well as skills that relate to storytelling/worldbuilding, organization, and artistic expression. This paper goes over the process I have taken in order to design, develop, and evaluate the game Watermelon Willy from the very start to alpha testing. The process started some time ago with the idea being present in my head for a while, however the project started with some basic storyboards to map out some of the fundamental aspects of the game. Coding began with a simple template, which was …

(Meta-)Physical Artworks: Digital Augmentation In Art Observation, Macy A. Toppan

Dartmouth College Master’s Theses

Augmented art— the subgenre of art that incorporates physical and digital artwork— is a rapidly growing field driven by advancing technology and a new generation for whom that tech is a given. Yet the presence of media like augmented and virtual reality in exhibition remains a controversial subject. Rather than focusing on the many theoretical debates about whether digital pieces can qualify as "good" art, we study it in practice through the eyes of the casual art observer. This paper highlights the audience in a within-participant study that asked viewers to take in a physical sculpture intentionally built with virtual …

Predicting Viral Rumors And Vulnerable Users With Graph-Based Neural Multi-Task Learning For Infodemic Surveillance, Xuan Zhang, Wei Gao

Research Collection School Of Computing and Information Systems

In the age of the infodemic, it is crucial to have tools for effectively monitoring the spread of rampant rumors that can quickly go viral, as well as identifying vulnerable users who may be more susceptible to spreading such misinformation. This proactive approach allows for timely preventive measures to be taken, mitigating the negative impact of false information on society. We propose a novel approach to predict viral rumors and vulnerable users using a unified graph neural network model. We pre-train network-based user embeddings and leverage a cross-attention mechanism between users and posts, together with a community-enhanced vulnerability propagation (CVP) …

Tracking People Across Ultra Populated Indoor Spaces By Matching Unreliable Wi-Fi Signals With Disconnected Video Feeds, Quang Hai Truong, Dheryta Jaisinghani, Shubham Jain, Arunesh Sinha, Jeong Gil Ko, Rajesh Krishna Balan

Research Collection School Of Computing and Information Systems

Tracking in dense indoor environments where several thousands of people move around is an extremely challenging problem. In this paper, we present a system — DenseTrack for tracking people in such environments. DenseTrack leverages data from the sensing modalities that are already present in these environments — Wi-Fi (from enterprise network deployments) and Video (from surveillance cameras). We combine Wi-Fi information with video data to overcome the individual errors induced by these modalities. More precisely, the locations derived from video are used to overcome the localization errors inherent in using Wi-Fi signals where precise Wi-Fi MAC IDs are used to …

Differences In Student-Ai Interaction Process On A Drawing Task: Focusing On Students' Attitude Towards Ai And The Level Of Drawing Skills, Jinhee Kim, Yoonhee Ham, Sang-Soog Lee

STEMPS Faculty Publications

Recent advances and applications of artificial intelligence (AI) have increased the opportunities for students to interact with AI in their learning tasks. Although various fields of scholarly research have investigated human-AI collaboration, the underlying processes of how students collaborate with AI in a student-AI teaming scenario have been scarcely investigated. To develop effective AI applications in education, it is necessary to understand differences in the student-AI interaction (SAI) process depending on students' characteristics. The present study attempts to fill this gap by exploring the differences in the SAI process amongst students with varying drawing proficiencies and attitudes towards AI in …

‘I Know It When I See It’– Developing Quality Schedules Considering Subjective Or Unspecified Criteria, Douglas L. Moody

Publications and Research

Most timetabling problems have a given objective function to measure the quality of a solution. However, users may have a “I know it when I see it” recognition of a quality schedule, without specifying the complete basis for their judgment. In this situation, the objective function cannot be exclusively used as a solution quality measurement. This work presents an AI based approach to aid in categorizing the solution’s quality when the users have not explicitly defined all factors used in their criteria.

Programming By Voice, Sadia Nowrin

Dissertations, Master's Theses and Master's Reports

Programmers typically rely on a keyboard and mouse for input, which poses significant challenges for individuals with motor impairments, limiting their ability
to effectively input programs. Voice-based programming offers a promising alternative,
enabling a more inclusive and accessible programming environment. Insights from interviews with motor-impaired programmers revealed that memorizing unnatural commands in existing voice-based programming systems led to frustration. In this work, we explore how programmers naturally speak a single line of code and present a comprehensive methodology for a voice programming system aimed at making programming more accessible for diverse users. To achieve this, we adopted a two-step pipeline. …

Random Walk Methods For Geometry Representation Agnostic Transport, Dario R. Seyb

Dartmouth College Ph.D Dissertations

In computer graphics, we use geometry representations to model a wide range of virtual scenes—from the fantastical worlds shown in animated movies to intricate mechanical parts.
These representations provide the context for transport problems—light transport is used to produce images of virtual scenes and diffusive transport to simulate distributions of quantities like heat.

There are many types of representations each with their own advantages.
For example, explicit ones make it easy to directly manipulate surfaces, while implicit representations allow for intuitive modeling by non-technical users and straightforward integration into machine learning systems.
Unfortunately, many algorithms that work on these digital …

Enhancing Students’ User Experience With A Code Critiquer, Laura E. Albrant

Dissertations, Master's Theses and Master's Reports

This thesis explores the role of human factors in the realm of code critiquers and students’ experiences with them. Across three studies, the work utilized Design Thinking to improve the user experience of WebTA for introductory engineering students learning MATLAB. The first two studies gathered observational and interview data to empathize, define, and ideate a new user interface (UI). Said UI was prototyped and then tested with the third study. Overall, the surveys’ data suggests that most students found the new design to be more appealing, useful, and purposeful; however, there is still plenty of room for improvement. Additionally, analysis …

The Impact Of Eye-Tracking On Mixed Reality Typing, Cecilia Schmitz

Dissertations, Master's Theses and Master's Reports

Accuracy and speed are pivotal when typing. We hypothesized that the lack of tactile feedback on midair mixed reality keyboards may adversely impact typing performance, especially when ten fingers are used to type. We examined the differences in performance when participants typed on a virtual keyboard using just their index fingers versus all ten fingers. The keyboard was deterministic (without auto-correct), relied only on the headset's egocentric cameras for sensing, and included symbol keys. We used a novel eye-tracking technique to mitigate accidental key presses. The technique was successful at reducing error rates, though participants still typed faster using their …

Tension Control And Interproximation Techniques Forshape Design And Rgb-Depth Segmentation Reconstruction And Modeling, Anastasia Kazadi

Theses and Dissertations--Computer Science

Human eyes possess remarkable capabilities to perceive and interpret a wealth of information about our environment; from discerning colors and depths to identifying object boundaries and navigating obstacles, our eyes serve as invaluable guides in our daily lives. Ongoing research in the fields of computer vision and computer graphics continuously explore the ways to replicate extraordinary human vision abilities in order to develop systems and frameworks which would enable computers to capture, analyze, and act upon discerned information. In this context, this dissertation seeks to investigate and automate various shape control and data processing techniques for 3D modeling and shape …

Garbage In ≠ Garbage Out: Exploring Gan Resilience To Image Training Set Degradations, Nicholas Crino, Bruce A. Cox, Nathan B. Gaw

Faculty Publications

Generative Adversarial Networks (GANs) have received immense attention in recent years due to their ability to capture complex, high-dimensional data distributions without the need for extensive labeling. Since their conception in 2014, a wide array of GAN variants have been proposed featuring alternative architectures, optimizers, and loss functions with the goal of improving performance and training stability. This manuscript focuses on quantifying the resilience of a GAN architecture to specific modes of image degradation. We conduct systematic experimentation to empirically determine the effects of 10 fundamental image degradation modes, applied to the training image dataset, on the Fréchet inception distance …

Human-Centered Machine Learning With Interpretable Visual Knowledge Discovery, Lincoln Huber

All Master's Theses

This research advances interpretable machine learning (ML) by introducing hyperblocks (HBs) as a structured, rule-based approach for creating transparent and accurate models using meaningful numeric attributes directly interpretable to end users. Key techniques, including Parallel Hyperblock Creation, Interactive Hyperblock Creation, Level n Hyperblock Creation, and k-Nearest Neighbor Hyperblock, provide a framework that ensures domain experts can meaningfully engage with the model’s decision-making process through lossless visualizations using General Line Coordinates (GLC). Case studies with the Wisconsin Breast Cancer and MNIST datasets demonstrated HBs' effectiveness in handling high-risk and complex classification tasks, offering interpretable accuracy that traditional models struggle to achieve. …

Dynamic Meta-Path Guided Temporal Heterogeneous Graph Neural Networks, Yugang Ji, Chuan Shi, Yuan Fang

Research Collection School Of Computing and Information Systems

Graph Neural Networks (GNNs) have become the de facto standard for representation learning on topological graphs, which usually derive effective node representations via message passing from neighborhoods. Although GNNs have achieved great success, previous models are mostly confined to static and homogeneous graphs. However, there are multiple dynamic interactions between different-typed nodes in real-world scenarios like academic networks and e-commerce platforms, forming temporal heterogeneous graphs (THGs). Limited work has been done for representation learning on THGs and the challenges are in two aspects. First, there are abundant dynamic semantics between nodes while traditional techniques like meta-paths can only capture static …