Graphics and Human Computer Interfaces Commons^™

Susceptibility To High-Fidelity Misinformation: An Eye-Tracking Analysis, Yasasi Abeysinghe, Gavindya Jayawardena, Enkelejda Kasneci, Sampath Jayarathna

Computer Science Faculty Publications

With the rise of online misinformation and AI-generated text, understanding human perception of news truthfulness is critical. In this study, we examine visual attention and cognitive processing using eye-tracking measures as individuals read fake and real news articles sharing nearly identical structure and imagery, differing only in subtle textual changes. Using the public FakeNewsPerception dataset, we analyze advanced gaze measures, including scanpaths, AOI transitions, and luminance-corrected pupil measures, beyond basic gaze features, in relation to news truthfulness and perceived believability. Results show that, given the high fidelity of the fake news, readers exhibited comparable visual scanning patterns, attention allocation across …

Memebuddy: Dialog-Style Audio Representations For Engaging Non-Visual Meme Experiences, Chirag Bhansali, Vikas Ashok, Hae-Na Lee

Computer Science Faculty Publications

Image memes are a pervasive form of online communication, widely used to convey humor, opinions, and cultural references. Prior work has explored making memes accessible to blind users, primarily through auto-generated descriptive captions. While these approaches improve comprehensibility and sometimes incorporate prosodic or emotional cues, they often fail to capture the humor, narrative structure, and contextual nuances that make memes engaging. We present MemeBuddy, a system that models memes as dialog, generating structured, multi-turn audio representations using role-based speakers. MemeBuddy reinterprets a meme as a conversation between two speakers, integrating extracted meme text with contextual knowledge implicitly inferred by a …

Swimming In Uncertainty: Filling Data Gaps And Providing An Educational Platform For Beach Water Quality At Tybee Island, Georgia, Lukas Roberson

College of Graduate Studies: Theses & Dissertations

@font-face {font-family:"Cambria Math"; panose-1:2 4 5 3 5 4 6 3 2 4; mso-font-charset:0; mso-generic-font-family:roman; mso-font-pitch:variable; mso-font-signature:-536870145 1107305727 0 0 415 0;}p.MsoNormal, li.MsoNormal, div.MsoNormal {mso-style-unhide:no; mso-style-qformat:yes; mso-style-parent:""; margin:0in; mso-pagination:widow-orphan; font-size:12.0pt; font-family:"Times New Roman",serif; mso-fareast-font-family:"Times New Roman";}.MsoChpDefault {mso-style-type:export-only; mso-default-props:yes; mso-font-kerning:0pt; mso-ligatures:none;}div.WordSection1 {page:WordSection1;}

Swimming in beaches water contaminated with high levels of bacteria can make you sick. Current monitoring at the public beaches on Tybee Island consists of weekly monitoring and enumeration of fecal indicator bacteria that takes 24 hours for results. If the number of bacteria exceed regulatory limits, a public health advisory is issued, and affected waters are retested until …

Designing Narrative-Based Ai Assistance For Sensemaking In Collaborative Environments: Case Studies In Education And Dementia Care, Dylan Edward Moore

Dartmouth College Ph.D Dissertations

This thesis addresses a gap in the human-computer interaction literature regarding the design, development, and evaluation of narrative-based AI assistance for collaborative, complex problem solving. I explore this design space through three case studies across the domains of education and dementia care. This work encompasses multi-year industry partnerships and longitudinal fieldwork, user-centered design, dataset curation, model training, and system evaluation.

Specifically, the first case study considers a story-based web platform for teaching AI literacy through peer-generated, personalized narrative scaffolding. Learners on the platform showed significant knowledge gains and other learning-related outcomes. To describe the novel design of this system, I …

From Physical Correlation To Emotional Connection: The Role Of Passive Haptics On Empathy In Virtual Reality, Jemely Robles

Dartmouth College Master’s Theses

Virtual reality is increasingly explored as a tool for cultivating empathy, and haptic feedback as a tool for enhancing immersion. This paper investigates the effects of combining the two.  Fifty-two participants experienced a custom-built VR scene in which a character was shown packing up a room. Participants were assigned to either a haptic condition, receiving passive haptic feedback corresponding to the character's actions, or a non-haptic control condition that did not receive any haptic input. Trait empathy was measured beforehand, and state empathy and engagement were measured after the experience. Thematic analysis was conducted on post-study interviews, and headset recordings …

Purified Zero-Shot Sketch-Based Image Retrieval, Yang Zhou, Jingru Yang, Jin Wang, Kaixiang Huang, Guodong Lu, Shengfeng He

Research Collection School Of Computing and Information Systems

Sketches, as a new solution in multimedia systems that can replace natural language, are characterized by sparse visual cues such as simple strokes that differ significantly from natural images containing complex elements such as background, foreground, and texture. This misalignment poses substantial challenges for zero-shot sketch-based image retrieval (ZS-SBIR). Prior approaches match sketches to full images and tend to overlook redundant elements in natural images, leading to model distraction and semantic ambiguity. To address this issue, we introduce a distraction-agnostic framework, purified cross-domain matching (PuXIM), which operates on a straightforward principle: masking and matching. We devise a visual-cross-linguistic (VxL) sampler …

Generalized Visual Relation Detection With Diffusion Models, Kaifeng Gao, Siqi Chen, Hanwang Zhang, Jun Xiao, Yueting Zhuang, Qianru Sun

Research Collection School Of Computing and Information Systems

Visual relation detection (VRD) aims to identify relationships (or interactions) between object pairs in an image. Although recent VRD models have achieved impressive performance, they are all restricted to pre-defined relation categories, while failing to consider the semantic ambiguity characteristic of visual relations. Unlike objects, the appearance of visual relations is always subtle and can be described by multiple predicate words from different perspectives, e.g., “ride” can be depicted as “race” and “sit on”, from the sports and spatial position views, respectively. To this end, we propose to model visual relations as continuous embeddings, and design diffusion models to achieve …

Gui Test Migration Via Abstraction And Concretization, Yakun Zhang, Chen Liu, Xiaofei Xie, Yun Lin, Jin Song Dong, Dan Hao, Lu Zhang

Research Collection School Of Computing and Information Systems

GUI test migration aims to produce test cases with events and assertions to test specific functionalities of a target app. Existing migration approaches typically focus on the widget-mapping paradigm that maps widgets from source apps to target apps. However, since different apps may implement the same functionality in different ways, direct mapping may result in incomplete or buggy test cases, thus significantly impacting the effectiveness of testing the target functionality and the practical applicability of migration approaches.In this article, we propose a new migration paradigm (i.e., the abstraction-concretization paradigm) that first abstracts the test logic for the target functionality and …

Cross-Modal Proxy Evolving For Ood Detection With Vision-Language Models, Hao Tang, Yu Liu, Shuanglin Yan, Fei Shen, Shengfeng He, Jing Qin

Research Collection School Of Computing and Information Systems

Reliable zero-shot detection of out-of-distribution (OOD) inputs is critical for deploying vision-language models in open-world settings. However, the lack of labeled negatives in zero-shot OOD detection necessitates proxy signals that remain effective under distribution shift. Existing negative-label methods rely on a fixed set of textual proxies, which (i) sparsely sample the semantic space beyond in-distribution (ID) classes and (ii) remain static while only visual features drift, leading to cross-modal misalignment and unstable predictions. In this paper, we propose CoEvo, a training- and annotation-free test-time framework that performs bidirectional, sample-conditioned adaptation of both textual and visual proxies. Specifically, CoEvo introduces a …

Look, Compare And Draw: Differential Query Transformer For Automatic Oil Painting, Lingyu Liu, Yaxiong Wang, Li Zhu, Lizi Liao, Zhedong Zheng

Research Collection School Of Computing and Information Systems

This work introduces a new approach to automatic oil painting that emphasizes the creation of dynamic and expressive brushstrokes. A pivotal challenge lies in mitigating the duplicate and common-place strokes, which often lead to less aesthetic outcomes. Inspired by the human painting process, i.e., observing, comparing, and drawing, we incorporate differential image analysis into a neural oil painting model, allowing the model to effectively concentrate on the incremental impact of successive brushstrokes. To operationalize this concept, we propose the Differential Query Transformer (DQ-Transformer), a new architecture that leverages differentially derived image representations enriched with positional encoding to guide the stroke …