Graphics and Human Computer Interfaces Commons^™

Grokformer: Graph Fourier Kolmogorov‑Arnold Transformers, Guoguo Ai, Guansong Pang, Hezhe Qiao, Yuan Gao, Hui Yan

Research Collection School Of Computing and Information Systems

Graph Transformers (GTs) have demonstrated remarkable performance in graph representation learning over popular graph neural networks (GNNs). However, self-attention, the core module of GTs, preserves only low-frequency signals in graph features, leading to ineffectiveness in capturing other important signals like high-frequency ones. Some recent GT models help alleviate this issue, but their flexibility and expressiveness are still limited since the filters they learn are fixed on predefined graph spectrum or spectral order. To tackle this challenge, we propose a Graph Fourier Kolmogorov-Arnold Transformer (GrokFormer), a novel GT model that learns highly expressive spectral filters with adaptive graph spectrum and spectral …

Unambiguous Granularity Distillation For Asymmetric Image Retrieval, Hongrui Zhang, Yi Xie, Haoquan Zhang, Cheng Xu, Xuandi Luo, Donglei Chen, Xuemiao Xu, Huaidong Zhang, Pheng Ann Heng, Shengfeng He

Research Collection School Of Computing and Information Systems

Previous asymmetric image retrieval methods based on knowledge distillation have primarily focused on aligning the global features of two networks to transfer global semantic information from the gallery network to the query network. However, these methods often fail to effectively transfer local semantic information, limiting the fine-grained alignment of feature representation spaces between the two networks. To overcome this limitation, we propose a novel approach called Layered-Granularity Localized Distillation (GranDist). GranDist constructs layered feature representations that balance the richness of contextual information with the granularity of local features. As we progress through the layers, the contextual information becomes more detailed, …

Full-Stack Web Applications: Infrastructure, Development Pipelines & Devsecops, Yassine Chahid, Patrick Slattery

Publications and Research

This research explores emerging development methodologies and technologies which facilitate the deployment and maintenance of software applications. It evaluates architectural styles for the development of software such as monolithic (legacy) and microservice models, with a focus on their key differences such as scalability or project structure through to the development of an application. By examining methodologies such as Agile and continuous integration/continuous development pipelines along with the deployment tools Docker and Git for version/release control, the study analyzes how these innovations speed up development, improve existing practices, and serve as the foundation for development operations. Cloud solutions for tasks such …

Hdifftg: A Lightweight Hybrid Diffusion-Transformer-Gcn Architecture For 3d Human Pose Estimation, Yajie Fu, Chaorui Huang, Junwei Li, Hui Kong, Yibin Tian, Huakang Li, Zhiyuan Zhang

Research Collection School Of Computing and Information Systems

We propose HDiffTG, a novel 3D Human Pose Estimation (3DHPE) method that integrates Transformer, Graph Convolutional Network (GCN), and diffusion model into a unified framework. HDiffTG leverages the strengths of these techniques to significantly improve pose estimation accuracy and robustness while maintaining a lightweight design. The Transformer captures global spatiotemporal dependencies, the GCN models local skeletal structures, and the diffusion model provides step-by-step optimization for fine-tuning, achieving a complementary balance between global and local features. This integration enhances the model’s ability to handle pose estimation under occlusions and in complex scenarios. Furthermore, we introduce lightweight optimizations to the integrated model …

Efficient Prompt Tuning For Hierarchical Ingredient Recognition, Yinxuan Gui, Bin Zhu, Jingjing Chen, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

Fine-grained ingredient recognition presents a significant challenge due to the diverse appearances of ingredients, resulting from different cutting and cooking methods. While existing approaches have shown promising results, they still require extensive training costs and focus solely on fine-grained ingredient recognition. In this paper, we address these limitations by introducing an efficient prompt-tuning framework that adapts pretrained visual-language models (VLMs), such as CLIP, to the ingredient recognition task without requiring full model finetuning. Additionally, we introduce three-level ingredient hierarchies to enhance both training performance and evaluation robustness. Specifically, we propose a hierarchical ingredient recognition task, designed to evaluate model performance …

Action Dubber: Timing Audible Actions Via Inflectional Flow, Wenlong Wan, Weiying Zheng, Tianyi Xiang, Guiqing Li, Shengfeng He

Research Collection School Of Computing and Information Systems

We introduce the task of Audible Action Temporal Localization, which aims to identify the spatiotemporal coordinates of audible movements. Unlike conventional tasks such as action recognition and temporal action localization, which broadly analyze video content, our task focuses on the distinct kinematic dynamics of audible actions. It is based on the premise that key actions are driven by inflectional movements; for example, collisions that produce sound often involve abrupt changes in motion. To capture this, we propose T A2Net, a novel architecture that estimates inflectional flow using the second derivative of motion to determine collision timings without relying on audio …

Robust Relevance Feedback For Interactive Known-Item Video Search, Zhixin Ma, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

Known-item search (KIS) involves only a single search target, making relevance feedback-typically a powerful technique for efficiently identifying multiple positive examples to infer user intent-inapplicable. PicHunter addresses this issue by asking users to select the top-k most similar examples to the unique search target from a displayed set. Under ideal conditions, when the user's perception aligns closely with the machine's perception of similarity, consistent and precise judgments can elevate the target to the top position within a few iterations. However, in practical scenarios, expecting users to provide consistent judgments is often unrealistic, especially when the underlying embedding features used for …

Retrieval Augmented Generation For Dynamic Graph Modeling, Yuxia Wu, Lizi Liao, Yuan Fang

Research Collection School Of Computing and Information Systems

Modeling dynamic graphs, such as those found in social networks, recommendation systems, and e-commerce platforms, is crucial for capturing evolving relationships and delivering relevant insights over time. Traditional approaches primarily rely on graph neural networks with temporal components or sequence generation models, which often focus narrowly on the historical context of target nodes. This limitation restricts the ability to adapt to new and emerging patterns in dynamic graphs. To address this challenge, we propose a novel framework, Retrieval-Augmented Generation for Dy namic Graph modeling (RAG4DyG ), which enhances dynamic graph predictions by incorporating contextually and temporally relevant examples from broader …

Classification Of Human Trust In Ai Using Brain Activity Data, Danushka Bandara, Ruhuan Liao, Fatima Chowdhury, Leslie Abbott

Northeast Journal of Complex Systems (NEJCS)

Trust plays a crucial role in human-computer interaction, particularly in scenarios involving artificial intelligence (AI) systems. This study explores the feasibility of using functional near-infrared spectroscopy (fNIRS) data to classify trust levels in human-AI interaction scenarios. A total of 18 participants completed an image classification task with an AI team member while their hemodynamic responses were recorded using fNIRS. Preprocessing of fNIRS data involved motion artifact removal, filtering, and normalization. Exploratory analysis identified significant associations between hemodynamic responses in the prefrontal cortex and trust levels. An across-subject binary trust classification model was developed using machine learning techniques, achieving an F1 …

Multi-Level Differentiable Moving Particles With Partition Of Unity, Jinjin He

Dartmouth College Master’s Theses

Representing implicit geometry with intricate features has long been a challenge. Recent advances in Implicit Neural Representations (INRs) have shown great promise in applications such as 3D reconstruction, inverse rendering, and dynamic surface evolution. These methods leverage neural networks to model complex shapes continuously, offering advantages in resolution and flexibility over traditional discrete representations. Despite their success, efficiently handling fine geometric details and evolving dynamic scenes remains an open problem.

We introduce a differentiable moving particle representation based on the multi-level partition of unity (MPU) to model dynamic implicit geometries efficiently. Our approach employs two types of particles—feature particles and …

Tellings Of The Pacific Ocean: A Landscape-Based Approach For Multispecies Design And Hci, Maliheh Ghajargar

Engineering Faculty Articles and Research

Environmental disturbances induced by climate change have caused significant changes in our ecosystems and are threatening the health of our environments. As a response to this issue, a growing body of work has emerged in HCI and design, which seeks to foreground more-than-human stories in support of making more sustainable and just futures. This research contributes to this broad agenda by probing graphic novels as a multispecies storytelling method for design and HCI. Combining ideas from Anna Tsing’s adventures of landscape and from HCI and design’s use of sequential art (e.g., storyboards), we use landscape as the main protagonist of …

Nexusgs: Sparse View Synthesis With Epipolar Depth Priors In 3d Gaussian Splatting, Yulong Zheng, Zicheng Jiang, Shengfeng He, Yandu Sun, Junyu Dong, Huaidong Zhang, Yong Du

Research Collection School Of Computing and Information Systems

Neural Radiance Field (NeRF) and 3D Gaussian Splatting (3DGS) have noticeably advanced photo-realistic novel view synthesis using images from densely spaced camera viewpoints. However, these methods struggle in few-shot scenarios due to limited supervision. In this paper, we present NexusGS, a 3DGS-based approach that enhances novel view synthesis from sparse-view images by directly embedding depth information into point clouds, without relying on complex manual regularizations. Exploiting the inherent epipolar geometry of 3DGS, our method introduces a novel point cloud densification strategy that initializes 3DGS with a dense point cloud, reducing randomness in point placement while preventing over-smoothing and overfitting. Specifically, …

Hd-Epic: A Highly-Detailed Egocentric Video Dataset, Toby Perrett, Ahmad Darkhalil, Saptarshi Sinha, Omar Emara, Sam Pollard, Kranti Kumar Parida, Kaiting Liu, Prajwal Gatti, Siddhant Bansal, Kevin Flanagan, Jacob Chalk, Zhifan Zhu, Rhodri Guerrier, Fahd Abdelazim, Bin Zhu, Davide Moltisanti, Michael Wray, Hazel Doughty, Dima Damen

Research Collection School Of Computing and Information Systems

We present a validation dataset of newly-collected kitchenbased egocentric videos, manually annotated with highly detailed and interconnected ground-truth labels covering: recipe steps, fine-grained actions, ingredients with nutritional values, moving objects, and audio annotations. Importantly, all annotations are grounded in 3D through digital twinning of the scene, fixtures, object locations, and primed with gaze. Footage is collected from unscripted recordings in diverse home environments, making HDEPIC the first dataset collected in-the-wild but with detailed annotations matching those in controlled lab environments. We show the potential of our highly-detailed annotations through a challenging VQA benchmark of 26K questions assessing the capability to …

Dupin: A Parallel Framework For Densest Subgraph Discovery In Fraud Detection On Massive Graphs, Jiaxin Jiang, Siyuan Yao, Yuchen Li, Qiange Wang, Bingsheng He, Min Chen

Research Collection School Of Computing and Information Systems

Detecting fraudulent activities in financial and e-commerce transaction networks is crucial. One effective method for this is Densest Subgraph Discovery (DSD). However, deploying DSD methods in production systems faces substantial scalability challenges due to the predominantly sequential nature of existing methods, which impedes their ability to handle large-scale transaction networks and results in significant detection delays. To address these challenges, we introduce Dupin, a novel parallel processing framework designed for efficient DSD processing in billion-scale graphs. Dupin is powered by a processing engine that exploits the unique properties of the peeling process, with theoretical guarantees on detection quality and efficiency. …

A Digital Dive: Redesigning The Cabrillo High School Aquarium Website, Jacob V. Cacho

Graphic Communication

Tucked away on the Central Coast in Lompoc, you’ll find the Cabrillo High School (CHS) Aquarium. Started in 1986, the CHS Aquarium is the only high school aquarium of its kind in the nation run entirely by high school students. This 10,000+ square foot aquarium serves an underserved community at a Title I school, where students manage all aspects of animal care, nutrition, breeding, educational curriculum development, and visitor tours.

This program is truly one-of-a-kind and deserves the spotlight for just how unique it is. As a CHS graduate, I felt the current website lacked in many areas and could …

Community Detection In Heterogeneous Information Networks Without Materialization, Jiaxin Jiang, Siyuan Yao, Yuhang Chen, Bingsheng He, Yudong Niu, Yuchen Li, Shixuan Sun, Yongchao Liu

Research Collection School Of Computing and Information Systems

Community detection in heterogeneous information networks (HINs) poses significant challenges due to the diversity of entity types and the complexity of their interrelations. While traditional algorithms may perform adequately in some scenarios, many struggle with the high memory usage and computational demands of large-scale HINs. To address these challenges, we introduce a novel framework, SCAR, which efficiently uncovers community structures in HINs without requiring network materialization. SCAR leverages insights from meta-paths to interpret multi-relational data through compact vertex-based sketches, significantly reducing computational overhead and materialization overhead. We propose a sketch-based technique for estimating changes in modularity, improving both the precision …

Keep The Balance: A Parameter-Efficient Symmetrical Framework For Rgb+X Semantic Segmentation, Jiaxin Cai, Jingze Su, Qi Li, Wenjie Yang, Shu Wang, Tiesong Zhao, Shengfeng He, Wenxi Liu

Research Collection School Of Computing and Information Systems

Multimodal semantic segmentation is a critical challenge in computer vision, with early methods suffering from high computational costs and limited transferability due to full fine-tuning of RGB-based pre-trained parameters. Recent studies, while leveraging additional modalities as supplementary prompts to RGB, still predominantly rely on RGB, which restricts the full potential of other modalities. To address these issues, we propose a novel symmetric parameter-efficient fine-tuning framework for multimodal segmentation, featuring with a modality-aware prompting and adaptation scheme, to simultaneously adapt the capabilities of a powerful pre-trained model to both RGB and X modalities. Furthermore, prevalent approaches use the global cross-modality correlations …

Hvi: A New Color Space For Low-Light Image Enhancement, Qingsen Yan, Yixu Feng, Cheng Zhang, Guansong Pang, Kangbiao Shi, Peng Wu, Wei Dong, Jinqiu Sun, Yanning Zhang

Research Collection School Of Computing and Information Systems

Low-Light Image Enhancement (LLIE) is a crucial computer vision task that aims to restore detailed visual information from corrupted low-light images. Many existing LLIE methods are based on standard RGB (sRGB) space, which often produce color bias and brightness artifacts due to inherent high color sensitivity in sRGB. While converting the images using Hue, Saturation and Value (HSV) color space helps resolve the brightness issue, it introduces significant red and black noise artifacts. To address this issue, we propose a new color space for LLIE, namely Horizontal/Vertical-Intensity (HVI), defined by polarized HS maps and learnable inten sity. The former enforces …

Event-Based Eye Tracking: Event-Based Vision Workshop 2025, Qinyu Chen, Et. Al.

Research Collection School Of Computing and Information Systems

No abstract provided.

Modfinity: Unsupervised Domain Adaptation With Multimodal Information Flow Intertwining, Shanglin Liu, Jianming Lv, Jingdan Kang, Huaidong Zhang, Zequan Liang, Shengfeng He

Research Collection School Of Computing and Information Systems

Multimodal unsupervised domain adaptation leverages unlabeled data in the target domain to enhance multimodal systems continuously. While current state-of-the-art methods encourage interaction between sub-models of different modalities through pseudo-labeling and feature-level exchange, varying sample quality across modalities can lead to the propagation of inaccurate information, resulting in error accumulation. To address this, we propose Modal-Affinity Multimodal Domain Adaptation (MODfinity), a method that dynamically manages multimodal information flow through fine-grained control over teacher model selection, guiding information intertwining at both feature and label levels. By treating labels as an independent modality, MODfinity enables balanced performance assessment across modalities, employing a novel …