Graphics and Human Computer Interfaces Commons^™

Stylegan-∞: Extending Stylegan To Arbitrary-Ratio Translation With Stylebook, Yihua Dai, Tianyi Xiang, Bailin Deng, Yong Du, Hongmin Cai, Jing Qin, Shengfeng He

Research Collection School Of Computing and Information Systems

Although pre-trained large-scale generative models StyleGAN series have proven to be effective in various editing and translation tasks, they are limited to pre-defined fixed aspect ratio. To overcome this limitation, we propose StyleGAN-∞, a model that enables pre-trained StyleGAN to perform arbitrary-ratio conditional synthesis. Our key insight is to distill the expressive StyleGAN features into a StyleBook, such that an arbitrary-ratio condition can be translated to other forms by properly assembling pre-defined StyleBook vectors. To learn and leverage the StyleBook, we employ a network with three distinct stages, each corresponding to StyleBook extraction, StyleBook correspondence learning, and arbitrary-ratio synthesis. Extensive …

Map As A By-Product: Collective Landmark Mapping From Imu Data And User-Provided Texts In Situated Tasks, Ryo Yonetani, Kotaro Hara

Research Collection School Of Computing and Information Systems

This paper presents Collective Landmark Mapper, a novel map-as-a-by-product system for generating semantic landmark maps of indoor environments. Consider users engaged in situated tasks that require them to navigate these environments and regularly take notes on their smartphones. Collective Landmark Mapper exploits the smartphone's IMU data and the user's free text input during these tasks to identify a set of landmarks encountered by the user. The identified landmarks are then aggregated across multiple users to generate a unified map representing the positions and semantic information of all landmarks. In developing the proposed system, we focused specifically on retail applications and …

Towards Multimodal Emotional Support Conversation Systems, Yuqi Chu, Lizi Liao, Zhiyuan Zhou, Chong-Wah Ngo, Richang Hong

Research Collection School Of Computing and Information Systems

The integration of conversational artificial intelligence (AI) into mental health care promises a new horizon for therapist-client interactions, aiming to closely emulate the depth and nuance of human conversations. Despite the potential, the current landscape of conversational AI is markedly limited by its reliance on single-modal data, constraining the systems’ ability to empathize and provide effective emotional support. This limitation stems from a paucity of resources that encapsulate the multimodal nature of human communication essential for therapeutic counseling. To address this gap, we introduce the Multimodal Emotional Support Conversation (MESC) dataset, a first-of-its-kind resource enriched with comprehensive annotations across text, …

Vibemus: Proactive Agentic System For Music Personalization, Zhiliang Guo, Teng Tu, Yunshan Ma, Xun Yang

Research Collection School Of Computing and Information Systems

Large language models (LLMs) enable diverse forms of AI-assisted creation, yet they often struggle to bridge the preference-articulation gap: users may provide incomplete or vague intentions or lack the vocabulary to specify what they want, yielding outputs misaligned with true preferences. To address this gap and facilitate music creation in a vibe-centric environment, we introduce VibeMus, a proactive agentic system built on open-source components. The system engages in multi-turn dialogue to progressively determine the music’s emotion, genre, lyrics, and other aspects before generation. Simulated evaluations show that proactive clarification improves alignment with users’ intended nuances. Our approach is training-free, leveraging …

Robface: A Test Suite For Efficient Robustness Evaluation Of Face Recognition Systems, Ruihan Zhang, Jun Sun

Research Collection School Of Computing and Information Systems

Face recognition is a widely used authentication technology in practice, where robustness is required. It is thus essential to have an efficient and easy-to-use method for evaluating the robustness of (possibly third-party) trained face recognition systems. Existing approaches to evaluating the robustness of face recognition systems are either based on empirical evaluation (e.g., measuring attacking success rate using state-of-the-art attacking methods) or formal analysis (e.g., measuring the Lipschitz constant). While the former demands significant user efforts and expertise, the latter is extremely time-consuming. In pursuit of a comprehensive, efficient, easy-to-use, and scalable estimation of the robustness of face recognition systems, …

Towards Age-Inclusive Human Computer Interaction: A Study Of Text Message Adoption By Seniors, Sam Takavarasha, Varaidzo Mapepa

African Conference on Information Systems and Technology

Since mobile phones are increasingly becoming livelihoods-enablers to people in developing economies, inclusive interaction design are critical. This research on the age inclusivity of feature phones was motivated by some observation that elderly users had challenges with adoption of text messaging. We, therefore, hypothesized that elderly people had challenges with texting ‘cash’, ‘cheque’ or ‘visa’ on a feature phone. The paper investigates if the required agility and interface ergonomics were age discriminatory inhibitors of the adoption of text messaging given the diminishing dexterity and cognitive skills of seniors. Using Sen’s (1999) ‘heterogeneity of capabilities’ theory and Davis et al (1989)’s …

Creepy, Invasive, And Exploitative Algorithms: A Cpm Analysis Of Users' Privacy Breakdowns And Recalibration Practices With Social Media Algorithms, Matthew J. A. Craig, Jeffrey T. Child

Human-Machine Communication

Social media content filtering algorithms can both provide desired personalized content and ads for users. However, sometimes these recommendations can resemble individual private information. How might users navigate these experiences to best manage their private information? The present exploratory study utilizes the rules- and systems-based framework of communication privacy management (CPM) theory to explore social media users’ experiences of privacy breakdowns with social media algorithms and investigates what users do in response to said breakdowns. These responses were refined using content analysis and divided into different categories of privacy breakdowns and recalibration strategies. Implications for future research surrounding human-machine communication …

Machine Learning And Crime Prevention, Emily Lizewski

Student Theses

Predictive policing uses machine learning to analyze crime patterns and help law enforcement better efficient use their resources. These tools can improve accuracy by highlighting complex trends in large sets of data. While this technology has its advantages, it also raises important ethical and social questions. Within this paper we looks at how predictive policing works, focusing on the machine learning models often used such as decision trees, random forests, gradient boosting, and models that factor in both time and location. It also explores how these tools might unintentionally reinforce biases already present in historical crime data. In reviewing the …

Bhvit: Binarized Hybrid Vision Transformer, Tian Gao, Yu Zhang, Zhiyuan Zhang, Huajun Liu, Kaijie Yin, Chengzhong Xu, Hui Kong

Research Collection School Of Computing and Information Systems

Model binarization has made significant progress in enabling real-time and energy-efficient computation for con-volutional neural networks (CNN), offering a potential solution to the deployment challenges faced by Vision Transformers (ViTs) on edge devices. However, due to the structural differences between CNN and Transformer architectures, simply applying binary CNN strategies to the ViT models will lead to a significant performance drop. To tackle this challenge, we propose BHViT, a binarization-friendly hybrid ViT architecture and its full binarization model with the guidance of three important observations. Initially, BHViT utilizes the local information interaction and hierarchical feature aggregation technique from coarse to fine …

Quantizing Text-Attributed Graphs For Semantic-Structural Integration, Jianyuan Bo, Hao Wu, Yuan Fang

Research Collection School Of Computing and Information Systems

Text-attributed graphs (TAGs) have emerged as a powerful representation for modeling complex relationships across diverse domains. With the rise of large language models (LLMs), there is growing interest in leveraging their capabilities for graph learning. However, current approaches face significant challenges in embedding structural information into LLM-compatible formats, requiring either computationally expensive alignment mechanisms or manual graph verbalization techniques that often lose critical structural details. Moreover, these methods typically require labeled data from source domains for effective transfer learning, significantly constraining their adaptability. We propose STAG, a novel self-supervised framework that directly quantizes graph structural information into discrete tokens using …

Gcot: Chain-Of-Thought Prompt Learning For Graphs, Xingtong Yu, Chang Zhou, Zhongwei Kuai, Xinming Zhang, Yuan Fang

Research Collection School Of Computing and Information Systems

Chain-of-thought (CoT) prompting has achieved remarkable success in natural language processing (NLP). However, its vast potential remains largely unexplored for graphs. This raises an interesting question: How can we design CoT prompting for graphs to guide graph models to learn step by step? On one hand, unlike natural languages, graphs are non-linear and characterized by complex topological structures. On the other hand, many graphs lack textual data, making it difficult to formulate language-based CoT prompting. %Therefore we cannot directly adopt the CoT prompting methods used in the language domain. In this work, we propose the first CoT prompt learning framework …

Dreamanime: Learning Style-Identity Textual Disentanglement For Anime And Beyond, Chenshu Xu, Yangyang Xu, Huaidong Zhang, Xuemiao Xu, Shengfeng He

Research Collection School Of Computing and Information Systems

Text-to-image generation models have significantly broadened the horizons of creative expression through the power of natural language. However, navigating these models to generate unique concepts, alter their appearance, or reimagine them in unfamiliar roles presents an intricate challenge. For instance, how can we exploit language-guided models to transpose an anime character into a different art style, or envision a beloved character in a radically different setting or role? This paper unveils a novel approach named DreamAnime, designed to provide this level of creative freedom. Using a minimal set of 2-3 images of a user-specified concept such as an anime character …

Revolutionizing Digital Privacy Education For Older Adults: Enhanced Interventions And Ai-Assisted Learning Strategies, Heba Aly

All Dissertations

As older adults increasingly engage with digital platforms, they face unique privacy risks stemming from limited digital literacy, reduced trust in AI technologies, and constrained access—especially in rural or underserved communities. While digital tools offer benefits like social connection and information access, current privacy education efforts often neglect the needs of older adults. This dissertation addresses this gap by developing, testing, and refining digital privacy education interventions tailored for older adults, with a focus on trust, personalization, and AI-assisted learning.

Study 1 evaluates multiple instructional modalities across age groups, revealing older adults prefer structured videos and interactive tutorials, while younger …

Evometric: An Interactive Framework For Scalable Visual Analytics Of Time Series Data With Dynamic Changes., Jiahang Huang

Electronic Theses and Dissertations

In today's data-intensive landscape, rapid advances in digital sensing and recording technologies have enabled the acquisition of high-resolution multimodal time series data, capturing intricate real-world dynamics across various domains such as healthcare, behavioral science, and environmental monitoring. However, the complexity and scale of these datasets present significant analytical challenges, particularly in understanding dynamic changes at both individual and cohort levels. This dissertation introduces EvoMetric, a novel visual analytics framework designed to support scalable exploration and analysis of large-scale multimodal time series data with dynamic changes. EvoMetric seamlessly integrates individual-level temporal dynamics with population-level comparative insights, enabling users to visually …

Graph Positional Autoencoders As Self-Supervised Learners, Yang Liu, Deyu Bo, Wenxuan Cao, Yuan Fang, Yawen Li, Chuan Shi

Research Collection School Of Computing and Information Systems

Graph self-supervised learning seeks to learn effective graph representations without relying on labeled data. Among various approaches, graph autoencoders (GAEs) have gained significant attention for their efficiency and scalability. Typically, GAEs take incomplete graphs as input and predict missing elements, such as masked node features or edges. Although effective, our experimental investigation reveals that traditional feature or edge masking paradigms primarily capture low-frequency signals in the graph and fail to learn expressive structural information. To address these issues, we propose Graph Positional Autoencoders (GraphPAE), which employ a dual-path architecture to reconstruct both node features and positions. Specifically, the feature path …

Focus: Evaluating Pre-Trained Vision-Language Models On Underspecification Reasoning, Kankan Zhou, Yibin Lai, Kyriakos Mouratidis, Jing Jiang

Research Collection School Of Computing and Information Systems

Humans possess a remarkable ability to interpret underspecified ambiguous statements by inferring their meanings from contexts such as visual inputs. This ability, however, may not be as developed in recent pre-trained visionlanguage models (VLMs). In this paper, we introduce a novel probing dataset called FOCUS to evaluate whether state-of-the-art VLMs have this ability. FOCUS consists of underspecified sentences paired with image contexts and carefully designed probing questions. Our experiments reveal that VLMs still fall short in handling underspecification even when visual inputs that can help resolve the ambiguities are available. To further support research in underspecification, FOCUS will be released …

Affinitytune: A Prompt-Tuning Framework For Few-Shot Anomaly Detection On Graphs, Jingyan Chen, Guanghui Zhu, Guansong Pang, Chunfeng Yuan, Yihua Huang

Research Collection School Of Computing and Information Systems

Graph anomaly detection (GAD) is a critical task with applications in domains such as networking, finance, and bioinformatics. % However, the scarcity of labeled anomalies and the limitations of unsupervised methods hinder effective detection. % While semi-supervised and few-shot learning approaches offer improvements, they struggle with knowledge transfer and rely heavily on labeled data. % Recent advancements in prompt tuning on graphs provide a promising direction, but their application to heterophilous graphs in anomaly detection remains underexplored. % In this work, we propose AffinityTune, a novel framework for few-shot graph anomaly detection based on prompt tuning. % Our approach introduces …

Advancing Molecular Graph-Text Pre-Training Via Fine-Grained Alignment, Yibo Li, Yuan Fang, Mengmei Zhang, Chuan Shi

Research Collection School Of Computing and Information Systems

Understanding molecular structure and related knowledge is crucialfor scientific research. Recent studies integrate molecular graphswith their textual descriptions to enhance molecular representationlearning. However, they focus on the whole molecular graph andneglect frequently occurring subgraphs, known as motifs, whichare essential for determining molecular properties. Without suchfine-grained knowledge, these models struggle to generalize to un-seen molecules and tasks that require motif-level insights. To bridgethis gap, we propose FineMolTex, a novel Fine-grained Moleculargraph-Text pre-training framework to jointly learn coarse-grainedmolecule-level knowledge and fine-grained motif-level knowledge.Specifically, FineMolTex consists of two pre-training tasks: a con-trastive alignment task for coarse-grained matching and a maskedmulti-modal modeling task for …

Relightable Neural Radiance Fields For Novel View Synthesis, Malena I. Mahoney

Scholarly Horizons: University of Minnesota, Morris Undergraduate Journal

This paper describes relighting neural radiance fields for novel view synthesis. View synthesis is the problem of using input images with corresponding camera angles to produce a photorealistic 3D model of an environment and its objects. Neural radiance fields (NeRFs) were created as a solution to view synthesis. Neural radiance field models work well for generating realistic 3D models from 2D image inputs; how-ever, they do not support changing the lighting or placing the objects from the input images into different environments. The problem comes from the fact that NeRFs rely on a neural network that is essentially overfitted to …

Modern Procedural Terrain Generation Techniques And Their Background, Hunter A. Barton

2025 Symposium

Procedural terrain generation has become a staple in many digital environments, enabling the automated creation of large-scale and realistic landscapes for applications such as video games and movies. This paper provides an in-depth look at smooth noise functions and their use for terrain generation, as well as an overview of some more modern methods of generation. A method utilizing machine learning stlye transfer was reproduced for this paper with some alterations to improve visualization and realism.

Enhancing Graph Representation Learning Through Self-Supervision: An Augmentation Perspective, Jianyuan Bo

Dissertations and Theses Collection (Open Access)

Graph representation learning has become fundamental in various domains, from social networks to molecular structures, enabling extraction of meaningful patterns from graph-structured data. While deep learning approaches, particularly graph neural networks, have shown promising results, their effectiveness is often limited by the scarcity of labeled data. This challenge is particularly acute in graph domains where annotation requires specialized expertise and is prohibitively expensive. Self-supervised learning has emerged as a promising direction to address this limitation by creating auxiliary tasks from unlabeled data, with augmentation strategies playing a crucial role in their success.

Current graph self-supervised learning methods face several critical …

Efficient Prompt Tuning For Hierarchical Ingredient Recognition, Yinxuan Gui, Bin Zhu, Jingjing Chen, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

Fine-grained ingredient recognition presents a significant challenge due to the diverse appearances of ingredients, resulting from different cutting and cooking methods. While existing approaches have shown promising results, they still require extensive training costs and focus solely on fine-grained ingredient recognition. In this paper, we address these limitations by introducing an efficient prompt-tuning framework that adapts pretrained visual-language models (VLMs), such as CLIP, to the ingredient recognition task without requiring full model finetuning. Additionally, we introduce three-level ingredient hierarchies to enhance both training performance and evaluation robustness. Specifically, we propose a hierarchical ingredient recognition task, designed to evaluate model performance …

Experimental Analysis Of Satellite Operator Training Using Game-Based Virtual Reality Simulation, Lana Laskey

Doctoral Dissertations and Master's Theses

Satellite data plays a vital role in modern global infrastructure by enabling communications, navigation, and weather forecasting. As demand for satellite technology grows, so does the need for highly trained satellite ground operators. Traditional training regimens for satellite operators employ simulation using two-dimensional computer console displays paired with the varied ability of trainees to generate abstract mental imagery of the scenario. However, this development of mental imagery imposes a considerable learning curve and cognitive workload on the trainee, which may negatively impact the user experience and knowledge gained during the training scenario.

This experimental study investigated the effects of game-based …

Empowering Weight Loss: A Pragmatic Randomized Controlled Trial Of A Theory-Driven Self-Regulation Mobile App For Young Adults With Excess Body Weight, H. S. J. Chew, J. W. Ngooi, R. C. Du, P. Z. Chan, M. Jansson, B. Zhu, Y. Cao, Chong-Wah Ngo, R. Foo, A. Shabbir, D. Ho, N. Sevdalis, K. Y. Ngiam

Research Collection School Of Computing and Information Systems

Background/Introduction: Obesity is projected to affect more than half of the global population by 2035, posing significant health and economic challenges. While lifestyle modification is considered a cornerstone of weight management, its effectiveness often relies on substantial support systems. Purpose: This study aimed to evaluate the effectiveness of a 12-week, standalone Temporal Self-Regulation Theory (TST)-based weight loss mobile application, which integrates self-regulation techniques, food logging, and dietary nudging, in promoting weight loss among young adults with excess body weight. Methods: A two-arm, parallel-group, 1:1 randomized controlled trial was conducted, adhering to the CONSORT-Outcomes 2022 Extension guidelines. Participants completed a face-to-face …

Hdifftg: A Lightweight Hybrid Diffusion-Transformer-Gcn Architecture For 3d Human Pose Estimation, Yajie Fu, Chaorui Huang, Junwei Li, Hui Kong, Yibin Tian, Huakang Li, Zhiyuan Zhang

Research Collection School Of Computing and Information Systems

We propose HDiffTG, a novel 3D Human Pose Estimation (3DHPE) method that integrates Transformer, Graph Convolutional Network (GCN), and diffusion model into a unified framework. HDiffTG leverages the strengths of these techniques to significantly improve pose estimation accuracy and robustness while maintaining a lightweight design. The Transformer captures global spatiotemporal dependencies, the GCN models local skeletal structures, and the diffusion model provides step-by-step optimization for fine-tuning, achieving a complementary balance between global and local features. This integration enhances the model’s ability to handle pose estimation under occlusions and in complex scenarios. Furthermore, we introduce lightweight optimizations to the integrated model …

Advancing Food Nutrition Estimation Via Visual-Ingredient Feature Fusion, Huiyan Qi, Bin Zhu, Chong-Wah Ngo, Jingjing Chen, Ee-Peng Lim

Research Collection School Of Computing and Information Systems

Nutrition estimation is an important component of promoting healthy eating and mitigating diet-related health risks. Despite advances in tasks such as food classification and ingredient recognition, progress in nutrition estimation is limited due to the lack of datasets with nutritional annotations. To address this issue, we introduce FastFood, a dataset with 84,446 images across 908 fast food categories, featuring ingredient and nutritional annotations. In addition, we propose a new model-agnostic Visual-Ingredient Feature Fusion (VIF2 ) method to enhance nutrition estimation by integrating visual and ingredient features. Ingredient robustness is improved through synonym replacement and resampling strategies during training. The ingredient-aware …

Instruct2see: Learning To Remove Any Obstructions Across Distributions, Junhang Li, Yu Guo, Chuhua Xian, Shengfeng He

Research Collection School Of Computing and Information Systems

Images are often obstructed by various obstacles due to capture limitations, hindering the observation of objects of interest. Most existing methods address occlusions from specific elements like fences or raindrops, but are constrained by the wide range of real-world obstructions, making comprehensive data collection impractical. To overcome these challenges, we propose Instruct2See, a novel zero-shot framework capable of handling both seen and unseen obstacles. The core idea of our approach is to unify obstruction removal by treating it as a soft-hard mask restoration problem, where any obstruction can be represented using multi-modal prompts, such as visual semantics and textual instructions, …

Action Dubber: Timing Audible Actions Via Inflectional Flow, Wenlong Wan, Weiying Zheng, Tianyi Xiang, Guiqing Li, Shengfeng He

Research Collection School Of Computing and Information Systems

We introduce the task of Audible Action Temporal Localization, which aims to identify the spatiotemporal coordinates of audible movements. Unlike conventional tasks such as action recognition and temporal action localization, which broadly analyze video content, our task focuses on the distinct kinematic dynamics of audible actions. It is based on the premise that key actions are driven by inflectional movements; for example, collisions that produce sound often involve abrupt changes in motion. To capture this, we propose T A2Net, a novel architecture that estimates inflectional flow using the second derivative of motion to determine collision timings without relying on audio …

Unambiguous Granularity Distillation For Asymmetric Image Retrieval, Hongrui Zhang, Yi Xie, Haoquan Zhang, Cheng Xu, Xuandi Luo, Donglei Chen, Xuemiao Xu, Huaidong Zhang, Pheng Ann Heng, Shengfeng He

Research Collection School Of Computing and Information Systems

Previous asymmetric image retrieval methods based on knowledge distillation have primarily focused on aligning the global features of two networks to transfer global semantic information from the gallery network to the query network. However, these methods often fail to effectively transfer local semantic information, limiting the fine-grained alignment of feature representation spaces between the two networks. To overcome this limitation, we propose a novel approach called Layered-Granularity Localized Distillation (GranDist). GranDist constructs layered feature representations that balance the richness of contextual information with the granularity of local features. As we progress through the layers, the contextual information becomes more detailed, …

Grokformer: Graph Fourier Kolmogorov‑Arnold Transformers, Guoguo Ai, Guansong Pang, Hezhe Qiao, Yuan Gao, Hui Yan

Research Collection School Of Computing and Information Systems

Graph Transformers (GTs) have demonstrated remarkable performance in graph representation learning over popular graph neural networks (GNNs). However, self-attention, the core module of GTs, preserves only low-frequency signals in graph features, leading to ineffectiveness in capturing other important signals like high-frequency ones. Some recent GT models help alleviate this issue, but their flexibility and expressiveness are still limited since the filters they learn are fixed on predefined graph spectrum or spectral order. To tackle this challenge, we propose a Graph Fourier Kolmogorov-Arnold Transformer (GrokFormer), a novel GT model that learns highly expressive spectral filters with adaptive graph spectrum and spectral …