Graphics and Human Computer Interfaces Commons^™

Cookingdiffusion: Cooking Procedural Image Generation With Stable Diffusion, Yuan Wang, Bin Zhu, Yanbin Hao, Chong-Wah Ngo, Yi Tan, Xiang Wang

Research Collection School Of Computing and Information Systems

Recent advancements in text-to-image generation models have excelled in creating diverse and realistic images. This success extends to food imagery, where various conditional inputs like cooking styles, ingredients, and recipes are utilized. However, a yet-unexplored challenge is generating a sequence of procedural images based on cooking steps from a recipe. This could enhance the cooking experience with visual guidance and possibly lead to an intelligent cooking simulation system. To fill this gap, we introduce a novel task called cooking procedural image generation. This task is inherently demanding, as it strives to create photo-realistic images that align with cooking steps while …

Omnivton: Training-Free Universal Virtual Try-On, Zhaotong Yang, Yuhui Li, Shengfeng He, Xinzhe Li, Yangyang Xu, Junyu Dong, Yong Du

Research Collection School Of Computing and Information Systems

Image-based Virtual Try-On (VTON) techniques rely on either supervised in-shop approaches, which ensure high fidelity but struggle with cross-domain generalization, or unsupervised in-the-wild methods, which improve adaptability but remain constrained by data biases and limited universality. A unified, training-free solution that works across both scenarios remains an open challenge. We propose OmniVTON, the first training-free universal VTON framework that decouples garment and pose conditioning to achieve both texture fidelity and pose consistency across diverse settings. To preserve garment details, we introduce a garment prior generation mechanism that aligns clothing with the body, followed by continuous boundary stitching technique to achieve …

Exploring Object Status Recognition For Recipe Progress Tracking In Non-Visual Cooking, Franklin Mingzhe Li, Kaitlyn Ng, Bin Zhu, Patrick Carrington

Research Collection School Of Computing and Information Systems

Cooking plays a vital role in everyday independence and well-being, yet remains challenging for people with vision impairments due to limited support for tracking progress and receiving contextual feedback. Object status — the condition or transformation of ingredients and tools — offers a promising but underexplored foundation for context-aware cooking support. In this paper, we present OSCAR (Object Status Context Awareness for Recipes), a technical pipeline that explores the use of object status recognition to enable recipe progress tracking in non-visual cooking. OSCAR integrates recipe parsing, object status extraction, visual alignment with cooking steps, and time-causal modeling to support real-time …

Teaching Diffusion Models To Ground Alpha Matte, Tianyi Xiang, Weiying Zheng, Yutao Jiang, Tingrui Shen, Hewei Yu, Yangyang Xu, Shengfeng He

Research Collection School Of Computing and Information Systems

The power of visual language models is showcased in visual understanding tasks, where language-guided models achieve impressive flexibility and precision. In this paper, we ex tend this capability to the challenging domain of image matting by framing it as a soft grounding problem, enabling a single diffusion model to handle diverse objects, textures, and transparencies, all directed by descriptive text prompts. Our method teaches the diffusion model to ground alpha mattes by guiding it through a process of instance-level localization and transparency estimation. First, we introduce an intermediate objective that trains the model to accurately localize semantic components of the …

Stylegan-∞: Extending Stylegan To Arbitrary-Ratio Translation With Stylebook, Yihua Dai, Tianyi Xiang, Bailin Deng, Yong Du, Hongmin Cai, Jing Qin, Shengfeng He

Research Collection School Of Computing and Information Systems

Although pre-trained large-scale generative models StyleGAN series have proven to be effective in various editing and translation tasks, they are limited to pre-defined fixed aspect ratio. To overcome this limitation, we propose StyleGAN-∞, a model that enables pre-trained StyleGAN to perform arbitrary-ratio conditional synthesis. Our key insight is to distill the expressive StyleGAN features into a StyleBook, such that an arbitrary-ratio condition can be translated to other forms by properly assembling pre-defined StyleBook vectors. To learn and leverage the StyleBook, we employ a network with three distinct stages, each corresponding to StyleBook extraction, StyleBook correspondence learning, and arbitrary-ratio synthesis. Extensive …

Gti: Graph-Based Tree Index With Logarithm Updates For Nearest Neighbor Search In High-Dimensional Spaces, Ruoyao Ma, Yifan Zhu, Baihua Zheng, Lu Chen, Congcong Ge, Yunjun Gao

Research Collection School Of Computing and Information Systems

Nearest neighbor search (NNS) is fundamental for high-dimensional space retrieval and impacts various fields, such as pattern recognition, information retrieval, recommendation systems, and vector database management. Among existing NNS methods, graph-based methods often excel in query accuracy and efficiency. However, these methods face significant challenges, including high construction costs and difficulties with dynamic data updates. Recent efforts have focused on combining graph methods with hashing, quantization, and tree-based approaches to address these issues, but problems with large index sizes and update performance remain unresolved. In response, this paper proposes GTI, a novel, lightweight, and dynamic graph-based tree index for high-dimensional …

Deep Graph Anomaly Detection: A Survey And New Perspectives, Hezhe Qiao, Hanghang Tong, Nanyang Technological University, Irwin King, Charu Aggarwal, Guansong Pang

Research Collection School Of Computing and Information Systems

Graph anomaly detection (GAD), which aims to identify unusual graph instances (e.g., nodes, edges, subgraphs, or graphs), has attracted increasing attention in recent years due to its significance in a wide range of applications. Deep learning approaches, graph neural networks (GNNs) in particular, have been emerging as a promising paradigm for GAD, owing to its strong capability in capturing complex structure and/or node attributes in graph data. Considering the large number of methods proposed for GNN-based GAD, it is of paramount importance to summarize the methodologies and findings in the existing GAD studies, so that we can pinpoint effective model …

Map As A By-Product: Collective Landmark Mapping From Imu Data And User-Provided Texts In Situated Tasks, Ryo Yonetani, Kotaro Hara

Research Collection School Of Computing and Information Systems

This paper presents Collective Landmark Mapper, a novel map-as-a-by-product system for generating semantic landmark maps of indoor environments. Consider users engaged in situated tasks that require them to navigate these environments and regularly take notes on their smartphones. Collective Landmark Mapper exploits the smartphone's IMU data and the user's free text input during these tasks to identify a set of landmarks encountered by the user. The identified landmarks are then aggregated across multiple users to generate a unified map representing the positions and semantic information of all landmarks. In developing the proposed system, we focused specifically on retail applications and …

Vibemus: Proactive Agentic System For Music Personalization, Zhiliang Guo, Teng Tu, Yunshan Ma, Xun Yang

Research Collection School Of Computing and Information Systems

Large language models (LLMs) enable diverse forms of AI-assisted creation, yet they often struggle to bridge the preference-articulation gap: users may provide incomplete or vague intentions or lack the vocabulary to specify what they want, yielding outputs misaligned with true preferences. To address this gap and facilitate music creation in a vibe-centric environment, we introduce VibeMus, a proactive agentic system built on open-source components. The system engages in multi-turn dialogue to progressively determine the music’s emotion, genre, lyrics, and other aspects before generation. Simulated evaluations show that proactive clarification improves alignment with users’ intended nuances. Our approach is training-free, leveraging …

Robface: A Test Suite For Efficient Robustness Evaluation Of Face Recognition Systems, Ruihan Zhang, Jun Sun

Research Collection School Of Computing and Information Systems

Face recognition is a widely used authentication technology in practice, where robustness is required. It is thus essential to have an efficient and easy-to-use method for evaluating the robustness of (possibly third-party) trained face recognition systems. Existing approaches to evaluating the robustness of face recognition systems are either based on empirical evaluation (e.g., measuring attacking success rate using state-of-the-art attacking methods) or formal analysis (e.g., measuring the Lipschitz constant). While the former demands significant user efforts and expertise, the latter is extremely time-consuming. In pursuit of a comprehensive, efficient, easy-to-use, and scalable estimation of the robustness of face recognition systems, …

Towards Multimodal Emotional Support Conversation Systems, Yuqi Chu, Lizi Liao, Zhiyuan Zhou, Chong-Wah Ngo, Richang Hong

Research Collection School Of Computing and Information Systems

The integration of conversational artificial intelligence (AI) into mental health care promises a new horizon for therapist-client interactions, aiming to closely emulate the depth and nuance of human conversations. Despite the potential, the current landscape of conversational AI is markedly limited by its reliance on single-modal data, constraining the systems’ ability to empathize and provide effective emotional support. This limitation stems from a paucity of resources that encapsulate the multimodal nature of human communication essential for therapeutic counseling. To address this gap, we introduce the Multimodal Emotional Support Conversation (MESC) dataset, a first-of-its-kind resource enriched with comprehensive annotations across text, …

Towards Age-Inclusive Human Computer Interaction: A Study Of Text Message Adoption By Seniors, Sam Takavarasha, Varaidzo Mapepa

African Conference on Information Systems and Technology

Since mobile phones are increasingly becoming livelihoods-enablers to people in developing economies, inclusive interaction design are critical. This research on the age inclusivity of feature phones was motivated by some observation that elderly users had challenges with adoption of text messaging. We, therefore, hypothesized that elderly people had challenges with texting ‘cash’, ‘cheque’ or ‘visa’ on a feature phone. The paper investigates if the required agility and interface ergonomics were age discriminatory inhibitors of the adoption of text messaging given the diminishing dexterity and cognitive skills of seniors. Using Sen’s (1999) ‘heterogeneity of capabilities’ theory and Davis et al (1989)’s …

Creepy, Invasive, And Exploitative Algorithms: A Cpm Analysis Of Users' Privacy Breakdowns And Recalibration Practices With Social Media Algorithms, Matthew J. A. Craig, Jeffrey T. Child

Human-Machine Communication

Social media content filtering algorithms can both provide desired personalized content and ads for users. However, sometimes these recommendations can resemble individual private information. How might users navigate these experiences to best manage their private information? The present exploratory study utilizes the rules- and systems-based framework of communication privacy management (CPM) theory to explore social media users’ experiences of privacy breakdowns with social media algorithms and investigates what users do in response to said breakdowns. These responses were refined using content analysis and divided into different categories of privacy breakdowns and recalibration strategies. Implications for future research surrounding human-machine communication …

Machine Learning And Crime Prevention, Emily Lizewski

Student Theses

Predictive policing uses machine learning to analyze crime patterns and help law enforcement better efficient use their resources. These tools can improve accuracy by highlighting complex trends in large sets of data. While this technology has its advantages, it also raises important ethical and social questions. Within this paper we looks at how predictive policing works, focusing on the machine learning models often used such as decision trees, random forests, gradient boosting, and models that factor in both time and location. It also explores how these tools might unintentionally reinforce biases already present in historical crime data. In reviewing the …

Revolutionizing Digital Privacy Education For Older Adults: Enhanced Interventions And Ai-Assisted Learning Strategies, Heba Aly

All Dissertations

As older adults increasingly engage with digital platforms, they face unique privacy risks stemming from limited digital literacy, reduced trust in AI technologies, and constrained access—especially in rural or underserved communities. While digital tools offer benefits like social connection and information access, current privacy education efforts often neglect the needs of older adults. This dissertation addresses this gap by developing, testing, and refining digital privacy education interventions tailored for older adults, with a focus on trust, personalization, and AI-assisted learning.

Study 1 evaluates multiple instructional modalities across age groups, revealing older adults prefer structured videos and interactive tutorials, while younger …

Gcot: Chain-Of-Thought Prompt Learning For Graphs, Xingtong Yu, Chang Zhou, Zhongwei Kuai, Xinming Zhang, Yuan Fang

Research Collection School Of Computing and Information Systems

Chain-of-thought (CoT) prompting has achieved remarkable success in natural language processing (NLP). However, its vast potential remains largely unexplored for graphs. This raises an interesting question: How can we design CoT prompting for graphs to guide graph models to learn step by step? On one hand, unlike natural languages, graphs are non-linear and characterized by complex topological structures. On the other hand, many graphs lack textual data, making it difficult to formulate language-based CoT prompting. %Therefore we cannot directly adopt the CoT prompting methods used in the language domain. In this work, we propose the first CoT prompt learning framework …

Affinitytune: A Prompt-Tuning Framework For Few-Shot Anomaly Detection On Graphs, Jingyan Chen, Guanghui Zhu, Guansong Pang, Chunfeng Yuan, Yihua Huang

Research Collection School Of Computing and Information Systems

Graph anomaly detection (GAD) is a critical task with applications in domains such as networking, finance, and bioinformatics. % However, the scarcity of labeled anomalies and the limitations of unsupervised methods hinder effective detection. % While semi-supervised and few-shot learning approaches offer improvements, they struggle with knowledge transfer and rely heavily on labeled data. % Recent advancements in prompt tuning on graphs provide a promising direction, but their application to heterophilous graphs in anomaly detection remains underexplored. % In this work, we propose AffinityTune, a novel framework for few-shot graph anomaly detection based on prompt tuning. % Our approach introduces …

Evometric: An Interactive Framework For Scalable Visual Analytics Of Time Series Data With Dynamic Changes., Jiahang Huang

Electronic Theses and Dissertations

In today's data-intensive landscape, rapid advances in digital sensing and recording technologies have enabled the acquisition of high-resolution multimodal time series data, capturing intricate real-world dynamics across various domains such as healthcare, behavioral science, and environmental monitoring. However, the complexity and scale of these datasets present significant analytical challenges, particularly in understanding dynamic changes at both individual and cohort levels. This dissertation introduces EvoMetric, a novel visual analytics framework designed to support scalable exploration and analysis of large-scale multimodal time series data with dynamic changes. EvoMetric seamlessly integrates individual-level temporal dynamics with population-level comparative insights, enabling users to visually …

Bhvit: Binarized Hybrid Vision Transformer, Tian Gao, Yu Zhang, Zhiyuan Zhang, Huajun Liu, Kaijie Yin, Chengzhong Xu, Hui Kong

Research Collection School Of Computing and Information Systems

Model binarization has made significant progress in enabling real-time and energy-efficient computation for con-volutional neural networks (CNN), offering a potential solution to the deployment challenges faced by Vision Transformers (ViTs) on edge devices. However, due to the structural differences between CNN and Transformer architectures, simply applying binary CNN strategies to the ViT models will lead to a significant performance drop. To tackle this challenge, we propose BHViT, a binarization-friendly hybrid ViT architecture and its full binarization model with the guidance of three important observations. Initially, BHViT utilizes the local information interaction and hierarchical feature aggregation technique from coarse to fine …

Advancing Molecular Graph-Text Pre-Training Via Fine-Grained Alignment, Yibo Li, Yuan Fang, Mengmei Zhang, Chuan Shi

Research Collection School Of Computing and Information Systems

Understanding molecular structure and related knowledge is crucialfor scientific research. Recent studies integrate molecular graphswith their textual descriptions to enhance molecular representationlearning. However, they focus on the whole molecular graph andneglect frequently occurring subgraphs, known as motifs, whichare essential for determining molecular properties. Without suchfine-grained knowledge, these models struggle to generalize to un-seen molecules and tasks that require motif-level insights. To bridgethis gap, we propose FineMolTex, a novel Fine-grained Moleculargraph-Text pre-training framework to jointly learn coarse-grainedmolecule-level knowledge and fine-grained motif-level knowledge.Specifically, FineMolTex consists of two pre-training tasks: a con-trastive alignment task for coarse-grained matching and a maskedmulti-modal modeling task for …