Graphics and Human Computer Interfaces Commons^™

Quickque: Enabling Quick Access To Information In User Reviews For Screen Reader Users, Mohan Sunkara, Akshay Kolgar Nayak, Sandeep Kalari, Yash Prakash, Sampath Jayarathna, Hae-Na Lee, Vikas Ashok

Computer Science Faculty Publications

Efficiently perusing online customer reviews is presently challenging for blind users, who rely on a screen reader that supports predominantly one-dimensional narration of content via keyboard shortcuts. To address this, with restaurant reviews as seminal case study, we developed QuickCue, a browser extension prototype that enables screen reader users to quickly obtain the positives and negatives regarding different aspects of a restaurant (e.g., food quality, hygiene, ambiance), without having to sift through numerous reviews containing redundant information. At its core, QuickCue utilizes a large language model to perform aspect and sentiment-based joint classification of reviews to group them based on …

Automatic Scoring Cornhole Board System, Eric Diffendal, Jonah Harsh, Connor Lengel, Brett Sukie

Williams Honors College, Honors Research Projects

The objective is to create a self-scoring cornhole board that can detect and calculate each team's score based on the bags thrown each round and to be created at a low cost/eventually being sold at the current cost of a normal board. When playing cornhole, the game is simple: throw a bag on the board; however, the scores are variable (deduct and add) across each round. The most common issue when playing cornhole is miscalculations of the scores and forgetting the correct scores. Thus, this invention will make gameplay easy for all to play.

User Interface For Custom Car Infotainment Systems, Dylan Miller

Williams Honors College, Honors Research Projects

The infotainment system is often considered one of the most functional and luxurious aspects of modern cars, containing functions that are useful to drivers in ways that range from convenient to safety-enhancing. However, modern infotainment systems can have some drawbacks such as making it more difficult to repair the vehicles they are in, helping to artificially limit the lifespan of the vehicles they are in, and not being present in most vehicles more than 15 years old. The software described in this paper, OpenQarUI, seeks to be a part of a solution to these problems. It is a piece of …

Ultrasonic Sensor-Based Sound Synthesis Using Raspberry Pi Pico W, Niraj Jaishwal

Williams Honors College, Honors Research Projects

At the intersection of Human Computer Interaction and digital art, this project transforms simple motion into musical expression. It explores an interactive real-time sound synthesis system using ultrasonic sensors to generate continuous audio. The objective is to design a system that maps physical distances into musical parameters such as pitch and amplitude, which will create a responsive audio environment. Two ultrasonic sensors are used in combination with the Raspberry Pi Pico W microcontroller running CircuitPython and Adafruit Audio Hat for real-time sound output. One sensor controls the pitch of the generated tone, while the other controls volume. This enables expressive …

Personalized Physics Learning Through Ai: Insights From Problem Generation, Chatbot Dialogues, And Intelligent Tutoring Systems, Atharva Dange

Physics Dissertations - Archive

Artificial intelligence (AI) is poised to transform science education, yet questions remain on how best to integrate these technologies into teaching and learning. This dissertation investigates the use of AI-driven tools in university physics courses through three complementary studies. In the first study, a generative language model (ChatGPT) was used to create novel physics homework problems aligned with course objectives. Analysis showed that, after expert vetting, AI-generated questions can foster higher-order problem-solving and reduce student reliance on solution memorization, though careful instructor oversight is required to ensure accuracy. The second study embedded an AI chatbot as a learning aid in …

Polarimetric Capture And Differentiable Rendering, Katherine Anne Salesin

Dartmouth College Ph.D Dissertations

Many scientific fields rely on the capture and modeling of light to extract underlying information about the world. Often, more information can be extracted by capturing more about the nature of the light, such as its spectral shape or polarization state. While polarization is a relatively unexplored topic in computer graphics, when used in tandem with other recent advancements in the field it has enormous potential to improve both forward and inverse models in other scientific disciplines. We demonstrate this potential in two distinct settings in this thesis.

First, we apply the capture of polarized light to an inverse problem …

Optimizing Radial Interfaces For Eye-Movement Authentication On Smartphones, Trey V. Tuscai

Honors Theses

Radial authentication interfaces offer privacy-preserving, calibration-free eye-movement authentication. While their effectiveness has been demonstrated on large displays, their performance on smartphones remains underexplored. This study investigates seven radial interface configurations on the iPhone 13, varying the number of radial indicators and password lengths to examine trade-offs between accuracy, security, and entry time. Through a controlled eye-tracking experiment with 27 participants, we evaluate each configuration’s performance and collect user prioritizations of the three factors. Our findings reveal that shorter passwords with fewer indicators improve speed and accuracy but reduce security, while longer configurations enhance security at the cost of usability. Based …

Flowing Together Or Alone: Impact Of Collaboration In The Metaverse, Fiona Fui-Hoon Nah, Brenda Eschenbrenner, Langtao Chen

Research Collection School Of Computing and Information Systems

The metaverse is the next-generation Internet (Web3) that facilitates social connections and collaborations in a virtual world environment. Given the potential of the metaverse to provide more satisfying and effective means of remote collaborations, exploring the possibility of leveraging the metaverse for these endeavors is warranted. Therefore, an important question to address is whether greater engagement occurs when tasks are completed collaboratively versus individually in the metaverse. We address this question by drawing on flow and transportation theories to hypothesize the effect of carrying out a creative task in the metaverse collaboratively versus alone on one's cognitive absorption, a contextually …

Gnnsynergy: A Multi-View Graph Neural Network For Predicting Anti-Cancer Drug Synergy, Zhifeng Hao, Jianming Zhan, Yuan Fang, Min Wu, Ruichu Cai

Research Collection School Of Computing and Information Systems

Drug combinations play very important roles in cancer therapy, as they can enhance curative efficacy and overcome drug resistance. Due to the increasing size of combinatorial space, experimental screening for all the drug combinations becomes infeasible in practice. Therefore, there is a great need to develop accurate computational approaches that can predict potential drug combinations to direct the experimental screening. In this paper, we propose a novel method called GNNSynergy to learn drug embeddings for drug synergy prediction. Given a specific cancer cell line, we propose a multi-view graph neural network framework which considers the current cell line as main …

Recdreamer: Consistent Text-To-3d Generation Via Uniform Score Distillation, Chenxi Zheng, Yihong Lin, Bangzhen Liu, Xuemiao Xu, Yongwei Nie, Shengfeng He

Research Collection School Of Computing and Information Systems

Current text-to-3D generation methods based on score distillation often suffer from geometric inconsistencies, leading to repeated patterns across different poses of 3D assets. This issue, known as the Multi-Face Janus problem, arises because existing methods struggle to maintain consistency across varying poses and are biased toward a canonical pose. While recent work has improved pose control and approximation, these efforts are still limited by this inherent bias, which skews the guidance during generation. To address this, we propose a solution called RecDreamer, which reshapes the underlying data distribution to achieve more consistent pose representation. The core idea behind our method …

Synthesizing Multi-Person And Rare Pose Images For Human Pose Estimation, Liuqing Zhao, Zichen Tian, Zou Peng, Richang Hong, Qianru Sun

Research Collection School Of Computing and Information Systems

Human pose estimation (HPE) models underperform in recognizing rare poses because they suffer from data imbalance problems (i.e., there are few image samples for rare poses) in their training datasets. From a data perspective, the most intuitive solution is to synthesize data for rare poses. Specifically, the rule-based methods apply manual manipulations (such as Cutout and GridMask) to the existing data, so the limited diversity of the data constrains the model. An alternative method is to learn the underlying data distribution via deep generative models (such as ControlNet and HumanSD) and then sample “new data” from the distribution. This works …

Accessmenu: Enhancing Usability Of Online Restaurant Menus For Screen Reader Users, Nithiya Venkatraman, Akshay Kolgar Nayak, Suyog Dahal, Yash Prakash, Hae-Na Lee, Vikas Ashok

Computer Science Faculty Publications

Online food ordering has become commonplace due to its convenience. The wide variety of culinary choices, combined with fast and economical door-delivery services, encourages more people to order food online. To facilitate this process, food vendors, including restaurants, often provide full menus on their websites, typically in visual formats such as images or PDFs. While this is convenient for sighted users, blind and visually impaired (BVI) individuals face significant challenges accessing these visual menus with their screen reader assistive technology. An interview study with 12 BVI screen reader users revealed that present assistive tools do not adequately satisfy the needs …

Automation To Autonomy: Temporal Dynamics Of Trust And Visual Attention Allocation Did Not Evolve, Tetsuya Sato, Eric Chancey, Yusuke Yamani

Psychology Faculty Publications

Emerging work environments are expected to implement autonomy that performs various functions without human input. Previous works has shown that trust in automation is negatively correlated with visual attention allocation, indicating that trust is a dynamic construct. Moreover, trust in automation and trust in autonomy appears to evolve in similar ways. However, recent work has demonstrated differences between trust in automation and trust in autonomy within Kaber’s (2018) theoretical framework (Sato et al., 2023b). Yet, it is uncertain whether the development of trust and visual attention allocation differs between automation and autonomy. The present study examined the temporal dynamics of …

The Interacting Roles Of Attention Allocation And Trust In Highly Automated Aam Environments, Yusuke Yamani

Psychology Faculty Publications

[First slide]

Mechanisms of attentive visual processing

• Attention control
• Visual search
• Eye movement
• Aging and individual differences

Limits of human performance in applied environment

• Complex displays
• Machine operation
• Surface transportation
• Advanced air mobility
• Nuclear operation

Methods to ameliorate human cognitive performance

• Human-machine interface
• Human autonomy/AI teaming
• Human-systems integration
• Training

Human Perception Of Ai Capabilities At Classifying Perturbed Roadway Signs, Katherine R. Garcia, Jing Chen, Yanru Xiao, Scott Mishler, Cong Wang, Bin Hu

Computer Science Faculty Publications

Artificial Intelligence (AI) is crucial to numerous functions required for driving automation systems, including the computer vision techniques used to detect the roadway environment and make real-time decisions. However, the images used as inputs to the AI system may be maliciously perturbed, or manipulated, causing the AI system to make an incorrect classification. In this study, we examined humans’ perception of the AI’s computer vision capability of classifying various road sign images, including the original images, images with two different types of malicious attacks, and images that are scrambled randomly at the pixel level. Our results showed that participants rated …

A Real-Time Approach To Capture Ambient And Focal Attention In Visual Search, Gavindya Jayawardena, Yasith Jayawardana, Yasasi Abeysinghe, Bhanuka Mahanama, Sampath Jayarathna, Jacek Gwizdka

Computer Science Faculty Publications

During visual search, individuals’ attention shifts between ambient and focal states in response to task demands and stimuli. The ambient/focal coefficient K is a statistically validated measure of these states, computed offline from fixation duration and saccade amplitude data. While current methods compute K offline, real-time computation could enable applications such as monitoring user attention, creating attention-adaptive user interfaces, and optimizing graphics rendering. However, real-time computation of K requires stable estimates for the parameters of fixation duration and saccade amplitude distributions. Since these distributions are heavy-tailed, the real-time estimates exhibit high variance and slow convergence. To overcome this, we propose …

Optiselect And Enshap: Integrating Machine Learning And Game Theory For Ischemic Stroke Prediction, Pritam Chakraborty, Anjan Bandyopadhyay, Sricheta Parul, Sujata Swain, Partha Sarathy Banerjee, Tapas Si, Hong Qin, Saurav Mallik

Computer Science Faculty Publications

Stroke analysis using game theory and machine learning techniques. The study investigates the use of the Shapley value in predictive ischemic brain stroke analysis. Initially, preference algorithms identify the most important features in various machine learning models, including logistic regression, K-nearest neighbor, decision tree, support vector machine (linear kernel), support vector machine ( RBF kernel), neural networks, etc. For each sample, the top 3, 4, and 5 features are evaluated and selected to evaluate their performance. The Shapley value method was used to rank the models using their best four features based on their predictive capabilities. As a result, better-performing …

Insights In Adaptation: Examining Self-Reflection Strategies Of Job Seekers With Visual Impairments In India, Akshay Kolgar Nayak, Yash Prakash, Sampath Jayarathna, Hae-Na Lee, Vikas Ashok

Computer Science Faculty Publications

Significant changes in the digital employment landscape, driven by rapid technological advancements and the COVID-19 pandemic, have introduced new opportunities for blind and visually impaired (BVI) individuals in developing countries like India. However, a significant portion of the BVI population in India remains unemployed despite extensive accessibility advancements and job search interventions. Therefore, we conducted semi-structured interviews with 20 BVI persons who were either pursuing or recently sought employment in the digital industry. Our findings reveal that despite gaining digital literacy and extensive training, BVI individuals struggle to meet industry requirements for fulfilling job openings. While they engage in self-reflection …

Adapting Online Customer Reviews For Blind Users: A Case Study Of Restaurant Reviews, Mohan Sunkara, Akshay Kolgar Nayak, Sandeep Kalari, Yash Prakash, Sampath Jayarathna, Hae-Na Lee, Vikas Ashok

Computer Science Faculty Publications

Online reviews have become an integral aspect of consumer decision-making on e-commerce websites, especially in the restaurant industry. Unlike sighted users who can visually skim through the reviews, perusing reviews remains challenging for blind users, who rely on screen reader assistive technology that supports predominantly one-dimensional narration of content via keyboard shortcuts. In an interview study, we uncovered numerous pain points of blind screen reader users with online restaurant reviews, notably, the listening fatigue and frustration after going through only the first few reviews. To address these issues, we developed QuickCue assistive tool that performs aspect-focused sentiment-driven summarization to reorganize …

An Optimized Generalized Multi-Color Point Implicit Solver For Intel Gpus Using Oneapi Esimd, Joseph Wassell, Mohammad Zubair, Aaron Walden, Gabriel Nastac, Eric Nielsen, Timothée Ewart

Computer Science Faculty Publications

This paper presents an efficient implementation of a linear-solver kernel relevant to FUN3D, a suite of computational fluid dynamics software developed at NASA’s Langley Research Center. The linear solver is optimized for a range of block sizes commonly used in FUN3D. The implementation targets Aurora, the Argonne Leadership Computing Facility’s (ALCF) exascale machine featuring Intel Data Center Max 1550 GPUs. The linear solver’s performance is memory bandwidth-bound due to its low arithmetic intensity. The primary performance challenges stem from variable matrix row lengths and indirect memory access patterns inherent in unstructured-grid applications. Variable block sizes introduce additional complexity through differing …

Surveying The Role Of Visual Analytics In Human-Machine Teaming, Naga Datha Saikiran Battula

Theses

Humans and machines both possess their unique capabilities and have their strengths and weaknesses, which can be complementary to one another and allow them to achieve a common goal. Teaming in the modern era involves text prompts, voice commands, gesture recognition, touch interfaces, and the latest visualization techniques that allow parties/agents to interact. Communication through visualization plays a vital role in allowing robust insights to be gained through a glance. Using visualization as a medium between humans and machines can increase the communication bandwidth. Human-machine teaming has witnessed much progress, with many theories and practical examples emerging. In the report, …

La Creatividad En Peligro: Como La Inteligencia Artificial Es Un Reto Para Los Artistas., Nathaly Cisneros

Capstones

Los artistas digitales han creado obras maestras que nos han dejado sin aliento con sus pinceles digitales, lápices y pinturas. Desde retratos que parecen saltar de la pantalla hasta paisajes que nos transportan a mundos desconocidos, su arte ha sido una fuente constante de inspiración.

Pero en los últimos años, una nueva fuerza ha comenzado a cambiar el juego. La inteligencia artificial ha estado avanzando a pasos agigantados y ahora se perfila como una amenaza para el futuro de los artistas digitales. ¿Qué significa esto para el arte y la creatividad?

Link: https://docs.google.com/document/d/1xe8UxDMekX_SwiIppyt_JppK8M-lB-YWNWGyeyShlJM/edit?usp=sharing

Real-Time Feedback-Driven Framework For Automated Cybersickness Mitigation, Md Jahirul Islam

Master's Theses

As technologies are becoming more advanced day by day, the embracement of virtual reality (VR) technology among users is also increasing in daily activities for various purposes, and subsequently, the barrier between the real and virtual world is fading. Despite the versatile uses, cybersickness (CS) is a major problem which is induced among users due to the immersive VR experience. There is a plethora of research findings and methods to measure the users’ CS such as virtual reality sickness questionnaire (VRSQ), simulator sickness questionnaire (SSQ), fast motion scale questionnaire (FMS), and others. Recently, machine learning approaches have also been adopted …

On The Benefits Of Directness In Virtual Characters For Motivational Interviews, Michael O'Mahony, Cathy Ennis, Robert Ross

Conference papers

Understanding the factors influencing successful engagement with Embodied Conversational Agents (ECAs) remains a significant challenge. This understanding could be used to personalise agents to users to improve interactions. Some studies have shown that simulating personalities in healthcare agents can improve effectiveness and engagement. However, it is not yet well understood how variations of agent personality can be leveraged to improve user engagement with Motivational Interviewing (MI) ECAs. Specifically how the balance between agent warmth and directness can be controlled in an MI agent to improve likeability and engagement. We conducted an online Wizard-of-Oz (WoZ) mediated study of two variants of …

Visualization Of Paleocurrents On A Web Application Using Gplates, Anjan Sapkota

MS in Computer Science Theses

Paleocurrents are flow directions derived from features of sedimentary rocks that reveal the direction of the current of wind or water that deposited the sediment. In 2015, Brand et al. created a global database of paleocurrents, which contains over 1,000,000 measurements worldwide: North America, South America, Australia, Great Britain, parts of Western Europe, China, Africa are fairly well represented; Antarctica, Eastern Europe, and Asia are modestly represented and Russia is poorly represented. The contribution of this thesis is a web application that uses the GPlates’ Application Programming Interface (API) to visualize global paleocurrents through time in an interactive way based …

Using Llms To Establish Implicit User Sentiment Of Software Desirability, Sherri Weitl-Harms, John D. Hastings, Jonah Lum

Research & Publications

This study explores the use of LLMs for providing quantitative zero-shot sentiment analysis of implicit software desirability, addressing a critical challenge in product evaluation where traditional review scores, though convenient, fail to capture the richness of qualitative user feedback. Innovations include establishing a method that 1) works with qualitative user experience data without the need for explicit review scores, 2) focuses on implicit user satisfaction, and 3) provides scaled numerical sentiment analysis, offering a more nuanced understanding of user sentiment, instead of simply classifying sentiment as positive, neutral, or negative.

Data is collected using the Microsoft Product Desirability Toolkit (PDT), …

Real-Time Motion Augmentation And Synthesis For Animating The Hands And Eyes Of Virtual Humans And Avatars, Ryan Canales

All Dissertations

Virtual Reality (VR) enables users to interact within virtual worlds via an embodied virtual representation of themselves called an “avatar”. Because avatars are essential for immersive experiences, it is important to consider how altering or augmenting avatar motion affects virtual experiences. This dissertation aims to improve virtual experiences by addressing some of the many challenges in animating avatars and virtual humans.

In our first study, we addressed the lack of tactile feedback during virtual grasping by using visual feedback techniques. We augmented the avatar’s hand motion to remain outside virtual objects (“outer hand”) even when the user’s hand penetrated them. …

Shifting Perspectives With Procedural Modeling Techniques And Vertex Animated Textures For Real-Time Interactive Morphing, Stephanie Schulze

All Theses

As humans experience reality, they intake external stimuli using sensory receptors to process information, forming a perception. Perceptions are subjective and shape each individual’s reality. Naturally then, by changing one’s perceptions, their experience of reality is altered, for better or worse. To emphasize the idea that perception is malleable and encourage a mindset of questioning alternative ways to look at a situation, an interactive experience in Unreal Engine is developed where a dull cityscape morphs and transforms into a surreal oversized nature scene. Procedural modeling techniques are used to create a variety of morphing assets, while Vertex Animated Textures are …

Innovations In Full-Stack Web Development: Front-End To Back-End, Yassine Chahid, Patrick Slattery

Publications and Research

This research explores emerging technologies within full-stack web development and their potential impact on current front-end and back-end solutions. Both areas employ crucial technologies that determine how end-users access information and navigate web services. Front-end solutions include HTML, JavaScript, and CSS which shape user interaction on websites. Back-end solutions use technologies such as SQL and PHP for the foundation of data processing, retrieval, and storage. The research method involves examining official documentation for these technologies to better understand their key components and to understand how their use in cyberspace has changed. The research will observe several high-traffic websites and domains …

Mvgamba : Unify 3d Content Generation As State Space Sequence Modeling, Xuanyu Yi, Zike Wu, Qiuhong Shen, Qingshan Xu, Pan Zhou, Joo-Hwee Lim, Shuicheng Yan, Xinchao Wang, Hanwang Zhang

Research Collection School Of Computing and Information Systems

Recent 3D large reconstruction models (LRMs) can generate high-quality 3D content in sub-seconds by integrating multi-view diffusion models with scalable multi-view reconstructors. Current works further leverage 3D Gaussian Splatting as 3D representation for improved visual quality and rendering efficiency. However, we observe that existing Gaussian reconstruction models often suffer from multi-view inconsistency and blurred textures. We attribute this to the compromise of multi-view information propagation in favor of adopting powerful yet computationally intensive architectures (e.g., Transformers). To address this issue, we introduce MVGamba, a general and lightweight Gaussian reconstruction model featuring a multi-view Gaussian reconstructor based on the RNN-like State …