Graphics and Human Computer Interfaces Commons^™

From Physical Correlation To Emotional Connection: The Role Of Passive Haptics On Empathy In Virtual Reality, Jemely Robles

Dartmouth College Master’s Theses

Virtual reality is increasingly explored as a tool for cultivating empathy, and haptic feedback as a tool for enhancing immersion. This paper investigates the effects of combining the two.  Fifty-two participants experienced a custom-built VR scene in which a character was shown packing up a room. Participants were assigned to either a haptic condition, receiving passive haptic feedback corresponding to the character's actions, or a non-haptic control condition that did not receive any haptic input. Trait empathy was measured beforehand, and state empathy and engagement were measured after the experience. Thematic analysis was conducted on post-study interviews, and headset recordings …

Mg-Spair: Multi-Grade Sparse-Guided Implicit Representation For Training-Data-Free Image Restoration, Jianmin Liao, Lei Huang, Ronglong Fang, Ashley Prater-Bennette, Lixin Shen, Yuesheng Xu

Mathematics & Statistics Faculty Publications

MG-SpaIR is a training-data-free framework for restoring a clean image from a single observation corrupted by a mixture of blur, downsampling, noise, and missing pixels. Building on implicit neural representations (INRs), we introduce a multi-grade residual hierarchy that progressively refines the reconstruction from low to high spatial frequencies across grades, improving representational fidelity and mitigating spectral limitations. To stabilize reconstruction optimization and suppress INR-induced artifacts, we further propose an explicit sparse proximal regularization (e.g., ℓ0 type) applied directly in the high-resolution image domain, which discourages spurious high-frequency patterns while preserving sharp structures. The resulting optimization is solved efficiently via a …

Cognitive Prosthetic: An Ai-Enabled Multimodal System For Episodic Recall In Knowledge Work, Lawrence Obiuwevwi, Krzystof J. Rechowicz, Vikas Ashok, Sachin Shetty, Sampath Jayarathna

Computer Science Faculty Publications

Modern knowledge workplaces increasingly strain human episodic memory as individuals navigate fragmented attention, overlapping meetings, and multimodal information streams. Existing workplace tools provide partial support through note-taking or analytics but rarely integrate cognitive, physiological, and attentional context into retrievable memory representations. This paper presents the Cognitive Prosthetic Multimodal System (CPMS)—an AI-enabled proof-of-concept designed to support episodic recall in knowledge work through structured episodic capture and natural language retrieval. CPMS synchronizes speech transcripts, physiological signals, and gaze behavior into temporally aligned, JSON-based episodic records processed locally for privacy. Beyond data logging, the system includes a web-based retrieval interface that allows users …

Voxvista: Enhancing Screen Reading Experience For Online User Comments, Yash Prakash, Akshay Kolgar Nayak, Mohammed Shoaib Alyaan, Sampath Jayarathna, Hae-Na Lee, Vikas Ashok

Computer Science Faculty Publications

Online discussions have become integral to how people exchange ideas, form opinions, and participate in collective deliberation. While sighted users can comfortably engage with online discussions, blind users who are dependent on screen readers are forced to listen to long threads narrated in a single, monotonic voice that lacks prosodic variation, rhythm, or emotion. This robotic auditory experience not only deteriorates the user engagement with the content but also increases cognitive strain, by making it difficult to remain attentive and discern meaning beyond literal words. In an interview study, most blind participants reported that monotonous narration hindered their ability to …

Modeling Joint Visual Attention In Naturalistic Dyadic Interactions, Kuushini Thennakoon, Yasasi Abeysinghe, Bhanuka Mahanama, Vikas Ashok, Sampath Jayarathna

Computer Science Faculty Publications

Joint visual attention (JVA) provides important insight into how individuals coordinate attention during social interaction. Egocentric eye tracking enables the study of JVA in natural, multi-user settings. This work presents a multi-stage framework to identify and analyze JVA using egocentric video and gaze data. The approach consists of three steps: spatiotemporal tube-based visual similarity, gaze-guided object detection, and attention pattern analysis using the ambient–focal coefficient K. Results show that object-focused collaborative activities exhibit high JVA, with object detection capturing higher joint attention than visual similarity, whereas conversation-based or independent activities show lower and more fragmented joint attention. Analysis of K …

Finding The Signal In The Noise: An Exploratory Study On Assessing The Effectiveness Of Ai And Accessibility Forums For Blind Users' Support Needs, Satwik Ram Kodandaram, Jiawei Zhou, Xiaojun Bi, Iv Ramakrishnan, Vikas Ashok

Computer Science Faculty Publications

Accessibility forums and, more recently, generative AI tools have become vital resources for blind users seeking solutions to computer-interaction issues and learning about new assistive technologies, screen reader features, tutorials, and software updates. Understanding user experiences with these resources is essential for identifying and addressing persistent support gaps. Towards this, we interviewed 14 blind users who regularly engage with forums and GenAI tools. Findings revealed that forums often overwhelm users with multiple overlapping topics, redundant or irrelevant content, and fragmented responses that must be mentally pieced together, increasing cognitive load. GenAI tools, while offering more direct assistance, introduce new barriers …

Lost In Instructions: Study Of Blind Users' Experiences With Diy Manuals And Ai-Rewritten Instructions For Assembly, Operation, And Troubleshooting Of Tangible Products, Monalika Padma Reddy, Aruna Balasubramanian, Jiawei Zhou, Xiaojun Bi, Iv Ramakrishnan, Vikas Ashok

Computer Science Faculty Publications

AI tools like ChatGPT and Be-My-AI are increasingly being used by blind individuals. Although prior work has explored their use in some Do-It-Yourself (DIY) tasks by blind individuals, little is known about how they use these tools and the available product-manual resources to assemble, operate, and troubleshoot physical/tangible products – tasks requiring spatial reasoning, structural understanding, and precise execution. We address this knowledge gap via an interview study and a usability study with blind participants, investigating how they leverage AI tools and product manuals for DIY tasks with physical products. Findings show that manuals are essential resources, but product-manual instructions …

Susceptibility To High-Fidelity Misinformation: An Eye-Tracking Analysis, Yasasi Abeysinghe, Gavindya Jayawardena, Enkelejda Kasneci, Sampath Jayarathna

Computer Science Faculty Publications

With the rise of online misinformation and AI-generated text, understanding human perception of news truthfulness is critical. In this study, we examine visual attention and cognitive processing using eye-tracking measures as individuals read fake and real news articles sharing nearly identical structure and imagery, differing only in subtle textual changes. Using the public FakeNewsPerception dataset, we analyze advanced gaze measures, including scanpaths, AOI transitions, and luminance-corrected pupil measures, beyond basic gaze features, in relation to news truthfulness and perceived believability. Results show that, given the high fidelity of the fake news, readers exhibited comparable visual scanning patterns, attention allocation across …

Memebuddy: Dialog-Style Audio Representations For Engaging Non-Visual Meme Experiences, Chirag Bhansali, Vikas Ashok, Hae-Na Lee

Computer Science Faculty Publications

Image memes are a pervasive form of online communication, widely used to convey humor, opinions, and cultural references. Prior work has explored making memes accessible to blind users, primarily through auto-generated descriptive captions. While these approaches improve comprehensibility and sometimes incorporate prosodic or emotional cues, they often fail to capture the humor, narrative structure, and contextual nuances that make memes engaging. We present MemeBuddy, a system that models memes as dialog, generating structured, multi-turn audio representations using role-based speakers. MemeBuddy reinterprets a meme as a conversation between two speakers, integrating extracted meme text with contextual knowledge implicitly inferred by a …

Gaze Transition Entropy And Automation Trust In Multitasking Workspace, Yusuke Yamani, Austin Jackson, Tetsuya Sato, Feyishola Ashimi, Michael S. Politowicz, Eric T. Chancey, Makoto Itoh

Psychology Faculty Publications

Safe flight operation requires visual scanning across multiple displays in a cockpit, which collectively represent the state of the aircraft and supporting automation. Trust is a crucial factor that drives human-automation interaction, and recent work has suggested a relationship between an operator's visual attention and automation trust. One index that captures predictability of eye movements between different areas of interest is gaze transition entropy. The current work reanalyzed data from Sato et al., which examined eye movement patterns and trust in automation associated with the system monitoring task of the Multi-Attribute Task Battery. Results showed credible positive correlations between the …

Can An Experienced Qualitative Researcher Distinguish Ai From Human Qualitative Content Analysis?, Alexandra T. Lucas, Jianna Ramos, Maria Bajwa, Aaron Calhoun, Mark W. Scerbo, Janice C. Palaganas

Psychology Faculty Publications

Background

Artificial intelligence (AI) has become increasingly embedded in research workflows. Large language models (LLMs) are being used to code segments of text, organise codes into themes and interpret patterns within contexts. Recent comparisons between human and AI analyses demonstrate up to 80% thematic overlap, yet humans consistently exhibit deeper interpretive integration and contextual understanding. This study assesses whether experienced researchers can distinguish between entirely human-generated and AI-generated qualitative content analyses of a simulation debriefing.

Methods

We conducted a qualitative descriptive study comparing human-generated qualitative content analysis (QCA) with ChatGPT-4o-generated QCA using a single focus group transcript on emotion management …

Error-Driven Density Control For Compact Gaussian Splatting Under Sparse Supervision, Abdelrhman Elrawy

Theses and Dissertations (Comprehensive)

This thesis studies efficiency and stability challenges in Gaussian-splatting-based reconstruction under sparse supervision. In few-shot novel view synthesis, standard 3D Gaussian Splatting (3DGS) can overfit the limited training views and grow an unnecessarily large number of primitives due to limitations in its Adaptive Density Control (ADC) mechanism. This thesis introduces an error-driven reformulation of ADC that triggers densification using opacity gradients as a lightweight proxy for rendering error, and shows that such aggressive densification must be paired with delayed and conservative pruning to prevent destructive create--destroy cycles. When combined with depth-based geometric regularization, the resulting framework produces substantially more compact …

Contextual Scaffolding And Self-Efficacy: Supporting Computer Skill Development Among Blind Learners In India, Akshay Kolgar Nayak, Yash Prakash, Sampath Jayarathna, Hae-Na Lee, Vikas Ashok

Computer Science Faculty Publications

Inclusive computer literacy education efforts, broadening the participation of blind or visually impaired (BVI) individuals, have gained traction in recent years. Existing literature investigating these efforts primarily draws evidence from affluent Global North contexts, where accessibility resources and legal frameworks are relatively more mature. Little is known about the in-situ teaching and learning challenges faced by trainers and BVI students, respectively, in resource-constrained, multicultural Global South countries like India. To address this knowledge gap, we conducted a four-month contextual inquiry at two computer training centers catering to 94 BVI students in India. We notably observed a rigid, experience-driven training environment …

Examining Inclusive Computing Education For Blind Students In India, Akshay Kolgar Nayak, Yash Prakash, Md Javedul Ferdous, Sampath Jayarathna, Hae-Na Lee, Vikas Ashok

Computer Science Faculty Publications

The growing demand for computer professionals, driven by the expanding Information Technology industry, has led to numerous inclusive computing education efforts. These efforts have even included blind or visually-impaired (BVI) students, who are being increasingly encouraged to pursue education and a career in computing, despite the visually-oriented nature of the discipline. Extant literature has predominantly focused on identifying and addressing the accessibility barriers faced by BVI students to promote more inclusive learning environments. While few studies have also investigated the accessibility of computing education from the perspectives of BVI learners and instructors, these have been primarily situated in the Global …

A Comparative Analysis Of Explainable Ai (Xai) Techniques For Transparent And Reliable Image Classification, Sovon Chakraborty, Shakib Mahmud Dipto, Kevin R. Pilkiewicz, Michael L. Mayo, Pratip Rana

Computer Science Faculty Publications

Evaluating the trustworthiness of black-box machine learning models remains a significant methodological challenge. Their lack of transparency and interpretability limits applicability, because stakeholders often seek transparency before trusting the results of black-box machine learning models. Explainable AI (XAI) methods provide for human-understandable justifications and informed decision-making of these black-box architectures. Therefore, it is imperative to select the proper XAI model tailored to specific tasks. In this research, we focus on examining four XAI techniques: PEEK, LRP, GRAD-CAM, and LIME to understand how they perform against each other for image classification tasks. We evaluate the performance, robustness, generalizability, noise stability, and …

Micro-Behavioral Analysis Of Online Shopping Patterns For Blind Users, Yash Prakash, Akshay Kolgar Nayak, Nithiya Venkatraman, Sampath Jayarathna, Hae-Na Lee, Vikas Ashok

Computer Science Faculty Publications

While online shopping platforms provide convenience and autonomy to blind users, their non-visual interactions remain underexplored at a micro-behavioral level. Existing studies have primarily emphasized accessibility and usability challenges but have overlooked how fine-grained, screen reader-driven keystroke-level behaviors reflect users’ cognitive strategies. In this paper, we present the findings of a longitudinal study with 25 blind participants to examine their micro-behavioral patterns, using keyboard activity and screen reader logs on both familiar and unfamiliar e-commerce websites. We complemented this study with semi-structured interviews to contextualize the uncovered micro-behavioral patterns. Our results revealed patterns in how blind users draw upon cognitive …

Geovig And Purevig: Geometry-Aware Architectures For Efficient Computer Vision, Omar Ismail

Theses and Dissertations (Comprehensive)

Deploying deep learning models for medical image analysis on mobile devices requires a balance between inference latency, memory footprint, and delineating anatomical boundaries with high accuracy. While Convolutional Neural Networks (CNNs) and mobile Vision Transformers (ViTs) offer efficiency, they often struggle to model the irregular, non-local geometric structures inherent in biological tissues without incurring prohibitive computational costs. In this thesis, we introduce GeoViG (Geometric Vision Graph), an architecture that bridges the gap between efficient grid-based processing and explicit Geometric Deep Learning. GeoViG introduces a novel transition from high-resolution pixel grids to low-resolution dynamic graphs via a SpreadEdgePool operator, a geometry-aware …

Trajlens: Visual Analysis For Constructing Cell Developmental Trajectories In Cross-Sample Exploration, Qipeng Wang, Shaolun Ruan, Rui Sheng, Yong Wang, Min Zhu, Huamin Qu

Research Collection School Of Computing and Information Systems

Constructing cell developmental trajectories is a critical task in single-cell RNA sequencing (scRNA-seq) analysis, enabling the inference of potential cellular progression paths. However, current automated methods are limited to establishing cell developmental trajectories within individual samples, necessitating biologists to manually link cells across samples to construct complete cross-sample evolutionary trajectories that consider cellular spatial dynamics. This process demands substantial human effort due to the complex spatial correspondence between each pair of samples. To address this challenge, we first proposed a GNN-based model to predict cross-sample cell developmental trajectories. We then developed TrajLens, a visual analytics system that supports biologists in …

Qualitative Study For Llm-Assisted Design Study Process: Strategies, Challenges, And Roles, Shaolun Ruan, Rui Sheng, Xiaolin Wen, Jiachen Wang, Tianyi Zhang, Yong Wang, Tim Dwyer, Jiannan Li

Research Collection School Of Computing and Information Systems

Design studies aim to develop visualization solutions for real-world problems across various application domains. Recently, the emergence of large language models (LLMs) has introduced new opportunities to enhance the design study process, providing capabilities such as creative problem-solving, data handling, and insightful analysis. However, despite their growing popularity, there remains a lack of systematic understanding of how LLMs can effectively assist researchers in visualization-specific design studies. In this paper, we conducted a rnulti-stage qualitative study to fill this gap, which involved 30 design study researchers from diverse backgrounds and expertise levels. Through in-depth interviews and carefully-designed questionnaires, we investigated strategies …

Cross-Modal Proxy Evolving For Ood Detection With Vision-Language Models, Hao Tang, Yu Liu, Shuanglin Yan, Fei Shen, Shengfeng He, Jing Qin

Research Collection School Of Computing and Information Systems

Reliable zero-shot detection of out-of-distribution (OOD) inputs is critical for deploying vision-language models in open-world settings. However, the lack of labeled negatives in zero-shot OOD detection necessitates proxy signals that remain effective under distribution shift. Existing negative-label methods rely on a fixed set of textual proxies, which (i) sparsely sample the semantic space beyond in-distribution (ID) classes and (ii) remain static while only visual features drift, leading to cross-modal misalignment and unstable predictions. In this paper, we propose CoEvo, a training- and annotation-free test-time framework that performs bidirectional, sample-conditioned adaptation of both textual and visual proxies. Specifically, CoEvo introduces a …