Graphics and Human Computer Interfaces Commons^™

The Future Is Now: Empowering Society Through Ai Literacy, Jason S. Wrench, Sanae Elmoudden

Milne Open Textbooks

Artificial Intelligence (AI) is no longer a futuristic concept—it is the reality of the present. From the algorithms shaping our social media feeds to the generative tools transforming our workplaces, AI has permeated every aspect of modern life. The Future is Now moves beyond the hype to provide a comprehensive roadmap for understanding, navigating, and shaping this technological revolution.

Demystifying the Machine

This textbook serves as a user-friendly guide to the “black box” of AI. It breaks down complex technical concepts—from machine learning and neural networks to large language models—making them accessible to students across all disciplines. By establishing a …

Visionglow: Evaluating Minimal-Disruption Smart-Home Control In Apple Vision Pro, Hongxiao Zheng

Dartmouth College Master’s Theses

Smart-home control in mixed-reality environments like Apple Vision Pro often relies on disruptive, application-based paradigms, such as using a smartphone or a windowed virtual interface. These methods create a “mode switch” that imposes cognitive load and pulls users from their primary tasks. We present VisionGlow, a minimal-disruption spatial interaction technique for Vision Pro. VisionGlow represents devices as spatially-anchored “orbs.” To control a device, the user looks at its orb and performs a pinch gesture, which invokes a compact, contextual control panel. We conducted a within-subjects study (N=18) comparing VisionGlow against two baselines: the standard Apple Home app on a smartphone …

Upright-Net+: Enhanced Learning Of Upright Orientation For 3d Point Clouds, Xufang Pang, Feng Li, Hongjie Zhuang, Ning Ding, Xiaopin Zhong, Shengfeng He, Wenxi Liu, Bo Jiang

Research Collection School Of Computing and Information Systems

Automatic 3D shape analysis is heavily influenced by the pose of input 3D models, as the continuous nature of pose space introduces complexities that usually exceed the encoding capacities of standard deep learning frameworks. To tackle this challenge, we present Upright-Net+, an enhancement of our previous model, Upright-Net, specifically developed for estimating upright orientation in 3D point clouds. Our approach is grounded in the design principle that ”form ever follows function,” treating the natural base of an object as a functional structure that stabilizes it in its typical pose, influenced by physical laws and geometric properties. We reformulate the continuous …

Jury-And-Judge Chain-Of-Thought For Uncovering Toxic Data In 3d Visual Grounding, Kaixiang Huang, Qifeng Zhang, Jin Wang, Jingru Yang, Yang Zhou, Huan Yu, Guodong Lu, Shengfeng He

Research Collection School Of Computing and Information Systems

3D Visual Grounding (3DVG) faces persistent challenges due to coarse scene-level observations and logically inconsistent annotations, which introduce ambiguities that compromise data quality and hinder effective model supervision. To address these challenges, we introduce Refer-Judge, a novel framework that harnesses the reasoning capabilities of Multimodal Large Language Models (MLLMs) to identify and mitigate toxic data. At the core of Refer-Judge is a Jury-and-Judge Chain-of-Thought paradigm, inspired by the deliberative process of the judicial system. This framework targets the root causes of annotation noise: jurors collaboratively assess 3DVG samples from diverse perspectives, providing structured, multi-faceted evaluations. Judges then consolidate these insights …

Cssa-Fusion: Channel Selective And Spatial Alignment Infrared-Visible Image Fusion, Zhen Li, Zhi Zeng, Zhongrui Xiao, Ming Wen, Zhiyuan Zhang, Yibin Tian

Research Collection School Of Computing and Information Systems

Infrared-visible image fusion aims to integrate complementary information from two modalities to generate images with enriched semantic content. However, existing methods often neglect two critical aspects: the design of a local–global feature enhancement architecture and spatial alignment. To address these challenges, we propose Channel Selective and Spatial Alignment Fusion (CSSA-Fusion), a novel framework composed of two synergistic modules. The first is a selective channel and redundancy suppression module, which introduces a dual-branch selective channel attention mechanism to jointly capture local saliency and global channel importance for enhanced feature representation, and an informativeness–redundancy separation strategy to suppress redundant information while preserving …

Implementation And Assessment Of The Openbci Platform As An Accessible Brain- Computer Interface, Jewell Norris

Honors Theses

OpenBCI is a low-cost, open-source platform for alternative brain-computer interface (BCI) software and hardware. This thesis evaluates OpenBCI’s electroencephalogram (EEG) and electromyography (EMG) capabilities by constructing and testing a 16-channel EEG system using the Ultracortex Mark IV headset and Cyton + Daisy biosensing board. The viability of the system was assessed through real-time BCI control and comparison to a clinical-grade EEG system. Real-time BCI control of an online falling-block game was tested via the use of eye blinks EMG (channels Fp1/Fp2) and head-tilt accelerometer inputs. The BCI game demonstrated reliable control despite minor latency and artifact sensitivity. For clinical comparison, …

Temporal And Heterogeneous Graph Neural Network For Remaining Useful Life Prediction, Zhihao Wen, Yuan Fang, Pengcheng Wei, Fayao Liu, Zhenghua Chen, Min Wu

Research Collection School Of Computing and Information Systems

Predicting remaining useful life (RUL) plays a crucial role in the prognostics and health management of industrial systems that involve a variety of interrelated sensors. Given a constant stream of time-series sensory data from such systems, deep learning (DL) models have risen to prominence at identifying complex, nonlinear temporal dependencies in these data. In addition to the temporal dependencies of individual sensors, spatial dependencies emerge as important correlations among these sensors, which can be naturally modeled by a temporal graph that describes time-varying spatial relationships. However, the majority of existing studies have relied on capturing discrete snapshots of this temporal …

Graph Perturbations For Robust Knowledge Discovery And Retrieval, Hanhua Xiao

Dissertations and Theses Collection (Open Access)

Graph perturbation, rooted in classical perturbation theory, studies how small topology edits, i.e., adding or deleting edges, affects graph properties (e.g., density, centrality). This fundamental problem underpins applications like bioinformatics, privacy preservation and system defense. While much prior work targets perturbations that influence global graph statistics or model outputs, comparatively little addresses robustness for knowledge discovery and information retrieval. In these settings, graphs are attributed: nodes carry real-world semantics (e.g., locations, people) and edges encode interactions or relationships. This thesis proposes new formulations and algorithms that generate and leverage graph perturbations to make knowledge discovery and retrieval more robust. Specifically, …

Addressing Sparsity For Knowledge Graph Completion: Data And Model Perspectives, Ran Liu

Dissertations and Theses Collection (Open Access)

Knowledge graphs (KGs) are powerful tools for structuring factual knowledge into relational triples, yet their practical utility is often adversely affected by data sparsity. Many entities and relations are associated with only a few observations, which limits the quality of learned embeddings and weakens generalization in downstream tasks. The problem of sparsity led to two interrelated challenges. Firstly, it restricts the informativeness of training samples: positive examples are scarce, and conventional negative sampling often produces trivial or redundant negatives that resulting in limited guidance. Secondly, in few-shot relation learning scenarios, sparsity worsens distribution shifts between training and test relations, as …

Seeing Culture: A Benchmark For Visual Reasoning And Grounding, Burak Satar, Zhixin Ma, Patrick Amadeus Irrawan, Wilfried Ariel Mulyawan, Jing Jiang, Ee-Peng Lim, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

Multimodal vision-language models (VLMs) have made substantial progress in various tasks that require a combined understanding of visual and textual content, particularly in cultural understanding tasks, with the emergence of new cultural datasets. However, these datasets frequently fall short of providing cultural reasoning while underrepresenting many cultures.In this paper, we introduce the Seeing Culture Benchmark (SCB), focusing on cultural reasoning with a novel approach that requires VLMs to reason on culturally rich images in two stages: i) selecting the correct visual option with multiple-choice visual question answering (VQA), and ii) segmenting the relevant cultural artifact as evidence of reasoning. Visual …

Instructors’ Strategies In Creating And Implementing Constructivist Llm-Based Learning Activities, Emily Aurelia, Shun Yi Yeo, Michelle Lui, Effie Lai-Chong Law, Anthony Tang

Research Collection School Of Computing and Information Systems

Large language models (LLMs) are increasingly being integrated into educational settings, enabling more adoption of constructivist teaching and learning approaches in classrooms. This paper explores the strategies instructors are currently using to incorporate LLMs into learning activities that align with constructivist principles, which emphasize that learners actively construct their own knowledge. Through interviews with nine instructors who have designed eleven distinct LLM-based activities and using reflexive thematic analysis, this study identifies various types of learning activities with respect to four different aspects of the constructivist learning theory. The strategies employed and challenges faced to foster constructivist student-LLM interaction were also …

Sketch-Sparsenet: Sparse Convolution Framework For Sketch Recognition, Jingru Yang, Jin Wang, Yang Zhou, Guodong Lu, Yu Sun, Huan Yu, Heming Fang, Zhihui Li, Shengfeng He

Research Collection School Of Computing and Information Systems

In free-hand sketch recognition, state-of-the-art methods often struggle to extract spatial features from sketches with sparse distributions, which are characterized by significant blank regions devoid of informative content. To address this challenge, we introduce a novel framework for sketch recognition, termed Sketch-SparseNet. This framework incorporates an advanced convolutional component: the Sketch-Driven Dilated Deformable Block (SD3B). This component excels at extracting spatial features and accurately recognizing free-hand sketches with sparse distributions. The SD3B component innovatively bridges gaps in the blank areas of sketches by establishing spatial relationships among disconnected stroke points through adaptive reshaping of convolution kernels. These kernels are deformable, …

Delving Into Invisible Semantics For Generalized One-Shot Neural Human Rendering, Yihong Lin, Xuemiao Xu, Huaidong Zhang, Cheng Xu, Weijie Li, Yi Xie, Jing Qin, Shengfeng He

Research Collection School Of Computing and Information Systems

Traditional human neural radiance fields often overlook crucial body semantics, resulting in ambiguous reconstructions, particularly in occluded regions. To address this problem, we propose the Super-Semantic Disentangled Neural Renderer (SSD-NeRF), which employs rich regional semantic priors to enhance human rendering accuracy. This approach initiates with a Visible-Invisible Semantic Propagation module, ensuring coherent semantic assignment to occluded parts based on visible body segments. Furthermore, a Region-Wise Texture Propagation module independently extends textures from visible to occluded areas within semantic regions, thereby avoiding irrelevant texture mixtures and preserving semantic consistency. Additionally, a view-aware curricular learning approach is integrated to bolster the model's …

Stroke2sketch: Harnessing Stroke Attributes For Training-Free Sketch Generation, Rui Yang, Huining Li, Yiyi Long, Xiaojun Wu, Shengfeng He

Research Collection School Of Computing and Information Systems

Generating sketches guided by reference styles requires precise transfer of stroke attributes, such as line thickness, deformation, and texture sparsity, while preserving semantic structure and content fidelity. To this end, we propose Stroke2Sketch, a novel training-free framework that introduces cross-image stroke attention, a mechanism embedded within self-attention layers to establish fine-grained semantic correspondences and enable accurate stroke attribute transfer. This allows our method to adaptively integrate reference stroke characteristics into content images while maintaining structural integrity. Additionally, we develop adaptive contrast enhancement and semanticfocused attention to reinforce content preservation and foreground emphasis. Stroke2Sketch effectively synthesizes stylistically faithful sketches that closely …

Cross-Subject Mind Decoding From Inaccurate Representations, Yangyang Xu, Bangzhen Liu, Wenqi Shao, Yong Du, Shengfeng He, Tingting Zhu

Research Collection School Of Computing and Information Systems

Decoding stimulus images from fMRI signals has advanced with pre-trained generative models. However, existing methods struggle with cross-subject mappings due to cognitive variability and subject-specific differences. This challenge arises from sequential errors, where unidirectional mappings generate partially inaccurate representations that, when fed into diffusion models, accumulate errors and degrade reconstruction fidelity. To address this, we propose the Bidirectional Autoencoder Intertwining framework for accurate decoded representation prediction. Our approach unifies multiple subjects through a Subject Bias Modulation Module while leveraging bidirectional mapping to better capture data distributions for precise representation prediction. To further enhance fidelity when decoding representations into stimulus images, …

Teaching Diffusion Models To Ground Alpha Matte, Tianyi Xiang, Weiying Zheng, Yutao Jiang, Tingrui Shen, Hewei Yu, Yangyang Xu, Shengfeng He

Research Collection School Of Computing and Information Systems

The power of visual language models is showcased in visual understanding tasks, where language-guided models achieve impressive flexibility and precision. In this paper, we ex tend this capability to the challenging domain of image matting by framing it as a soft grounding problem, enabling a single diffusion model to handle diverse objects, textures, and transparencies, all directed by descriptive text prompts. Our method teaches the diffusion model to ground alpha mattes by guiding it through a process of instance-level localization and transparency estimation. First, we introduce an intermediate objective that trains the model to accurately localize semantic components of the …

Stable Score Distillation, Haiming Zhu, Yangyang Xu, Chenshu Xu, Tingrui Shen, Wenxi Liu, Yong Du, Jun Yu, Shengfeng He

Research Collection School Of Computing and Information Systems

Text-guided image and 3D editing have advanced with diffusion-based models, yet methods like Delta Denoising Score often struggle with stability, spatial control, and editing strength. These limitations stem from reliance on complex auxiliary structures, which introduce conflicting optimization signals and restrict precise, localized edits. We introduce Stable Score Distillation (SSD), a streamlined framework that enhances stability and alignment in the editing process by anchoring a single classifier to the source prompt. Specifically, SSD utilizes Classifier-Free Guidance (CFG) equation to achieve cross-prompt alignment, and introduces a constant term null-text branch to stabilize the optimization process. This approach preserves the original content's …

Tactile Data Comics: Combining Step-By-Step Presentation Of Tactile Graphics With Verbal Narration For The Blind And Visually Impaired, Yang Jiao, Ruoting Sun, Rong Luo, Xiwen Yao, Xinran She, Kotaro Hara, Yuewen Zhang, Xinyi Fu

Research Collection School Of Computing and Information Systems

Tactile graphics on a refreshable display have proven effective in enabling visually impaired people to comprehend pictorial content. To further evaluate the effectiveness of refreshable tactile displays in blind education, we designed tactile data comics, a method that combines step-by-step presentation of tactile graphics with verbal narration. We conducted a user study with sixteen visually impaired students to compare tactile data comics against verbal-only and static tactile graphics. Our findings show that tactile data comics significantly improve participants’ comprehension and engagement during the learning experience. These empirical results suggest that the integration of refreshable tactile displays and tactile data comics …

Emoshortcuts: Emotionally Expressive Body Augmentation For Social Mixed Reality Avatars, Hyuna Seo, Youngki Lee, Rajesh Krishna Balan, Thivya Kandappu

Research Collection School Of Computing and Information Systems

We present EmoShortcuts1, a novel social Mixed Reality (MR) framework that enhances emotional expression by dynamically augmenting avatar body gestures to reflect users’ emotional states. While social MR enables immersive remote interactions through avatars, conveying emotions remains challenging due to limitations in head-mounted display (HMD) tracking (e.g., missing lower-body movements, such as stomping or defensive postures), and users’ tendency to deprioritize nonverbal expressions during multitasking. EmoShortcuts addresses these challenges by introducing an augmentation framework that generates expressive body gestures even when users’ physical movements are restricted. We conducted a formative study with 12 participants to identify key challenges in emotional …

Unresolved Image Simulation For Space Situational Awareness Applications, Fox Coniglario

Doctoral Dissertations and Master's Theses

The knowledge of what lies in orbit around Earth is at best a guess. Decades of spaceflight, debris buildup, and vehicle collisions have contributed to a large number of objects that are simply not able to be catalogued. Ongoing efforts to catalog debris in orbit have reached limits by conventional measures and as such, research is active in the field of in-orbit space situational awareness. This thesis intends to help fill a hole in the development of such orbital platforms by assisting the development of image processing software pipelines though the simulation of unresolved space imagery. The simulation uses accurate …