Open Access. Powered by Scholars. Published by Universities.®

Graphics and Human Computer Interfaces Commons™

Open Access. Powered by Scholars. Published by Universities.®

Research Collection School Of Computing and Information Systems

Discipline
Keyword
Publication Year

Articles 1 - 30 of 933

Full-Text Articles in Graphics and Human Computer Interfaces

Stprompt++: Prompting Vision-Language Models For Weakly Supervised Video Anomaly Detection And Fine-Grained Localization, Peng Wu, Chengyu Pan, Guansong Pang, Xiangteng He, Zhiwei Yang, Peng Wang, Yanning Zhang Jul 2028

Stprompt++: Prompting Vision-Language Models For Weakly Supervised Video Anomaly Detection And Fine-Grained Localization, Peng Wu, Chengyu Pan, Guansong Pang, Xiangteng He, Zhiwei Yang, Peng Wang, Yanning Zhang

Research Collection School Of Computing and Information Systems

Traditional weakly supervised video anomaly detection (WSVAD) tasks typically rely on coarse-grained frame-level labels for training. Although this approach reduces annotation costs, it results in weak semantic understanding and spatial localization capabilities due to the absence of fine-grained annotations, hindering precise pixel-level anomaly detection and localization. Thanks to the success of vision-language models (VLMs), e.g., CLIP, recent approaches leveraging large VLMs focus on exploiting their strong semantic understanding capabilities, but they typically feed only keyframes or short video segments into the models, without supplying sufficient prior contextual information (e.g., contextual frames around anomalies, zoomed-in anomaly regions, and detailed anomaly descriptions), …

Go to article

Generalizing Unsupervised Lidar Motion Estimation Via Patch-Aware Feature Reweighting Under Snowy Conditions, Beibei Zhou, Zhiyuan Zhang, Zhenbo Song, Jianhui Guo, Tongpo Zhang, Shaojing Song, Qinge Wu, Hui Kong Jan 2027

Generalizing Unsupervised Lidar Motion Estimation Via Patch-Aware Feature Reweighting Under Snowy Conditions, Beibei Zhou, Zhiyuan Zhang, Zhenbo Song, Jianhui Guo, Tongpo Zhang, Shaojing Song, Qinge Wu, Hui Kong

Research Collection School Of Computing and Information Systems

Unsupervised LiDAR motion estimation has attracted significant attention in pattern recognition and robotic vision. However, ensuring robust performance under adverse conditions — particularly structured noise like heavy snowfall — remains a challenging problem. Existing models struggle with domain generalization due to the sensitivity of feature extractors to spatial outliers, limiting their reliability in complex real-world scenarios. In this work, we present an unsupervised LiDAR motion estimation framework via patch-aware feature reweighting to mitigate the domain shift between clear and snowy conditions. Our approach prioritizes reliability-aware reweighting to reduce the influence of snowflake noise and outlier points on pose estimation, while …

Go to article

Drag As Evidence: Motion-Grounded Latent Recomposition For Drag-Based Editing, Xinyu Pu, Hongsong Wang, Jie Gui, Pan Zhou Dec 2026

Drag As Evidence: Motion-Grounded Latent Recomposition For Drag-Based Editing, Xinyu Pu, Hongsong Wang, Jie Gui, Pan Zhou

Research Collection School Of Computing and Information Systems

Modern image editors excel at semantic manipulation and visual synthesis, yet remain limited in precise spatial control, motivating the development of drag-based editing. However, existing drag-based methods often struggle to balance drag accuracy with natural, plausible, and intent-aligned generation. We propose MoReDrag, a motion-grounded drag-based editing method. Our key insight is to treat pixel-space warping as coarse motion evidence, and to inject this evidence into the generative sampling trajectory. Specifically, MoRe-Drag performs region-aware latent recomposition over refinement, inpainting, and anchor regions, coupled with stage-adaptive conditioning that progressively shifts from motion-grounded structure formation to semantic refinement. We further support an instruction-free …

Go to article

Defense-To-Attack: Bypassing Weak Defenses Enables Stronger Jailbreaks In Vision-Language Models, Yunhan Zhao, Xiang Zheng, Yige Li, Xingjun Ma Nov 2026

Defense-To-Attack: Bypassing Weak Defenses Enables Stronger Jailbreaks In Vision-Language Models, Yunhan Zhao, Xiang Zheng, Yige Li, Xingjun Ma

Research Collection School Of Computing and Information Systems

Despite their superb capabilities, Vision-Language Models (VLMs) have been shown to be vulnerable to jailbreak attacks. While recent jailbreaks have achieved notable progress, their effectiveness and efficiency can still be improved. In this work, we reveal an interesting phenomenon: incorporating weak defense cues into the attack pipeline can significantly enhance both the effectiveness and efficiency of jailbreaks on VLMs. Building on this insight, we propose Defense2Attack, a novel jailbreak method that bypasses the safety guardrails of VLMs by leveraging defensive patterns to guide jailbreak prompt construction. Specifically, Defense2Attack consists of three key components: (1) a visual optimizer that embeds universal …

Go to article

Characterizing And Repairing Obsolete Android Gui Tests Under Ui Evolution, Shiwen Song, Yiheng Xiong, Wenbo Guo, Manqi Sun, Jiaolong Kong, Xiaofei Xie Oct 2026

Characterizing And Repairing Obsolete Android Gui Tests Under Ui Evolution, Shiwen Song, Yiheng Xiong, Wenbo Guo, Manqi Sun, Jiaolong Kong, Xiaofei Xie

Research Collection School Of Computing and Information Systems

Graphical user interface (GUI) tests are widely used in regression testing of mobile applications (apps) tovalidate app behavior from the user’s perspective. However, frequent app evolution, such as UI redesignsand feature updates, often renders existing GUI tests obsolete, even when underlying functionality remainsunchanged. Automatically repairing such tests is critical for maintaining test suites and reducing substantialmanual effort. Despite the practical importance, there is still a lack of a systematic understanding of thecharacteristics of obsolete GUI tests and a publicly available benchmark to support their study. To fill thisgap, we construct a benchmark comprising 736 obsolete GUI tests collected from 36 …

Go to article

Restoring Linguistic Grounding In Vla Models Via Train-Free Attention Recalibration, Ninghao Zhang, Bin Zhu, Shijie Zhou, Jingjing Chen Sep 2026

Restoring Linguistic Grounding In Vla Models Via Train-Free Attention Recalibration, Ninghao Zhang, Bin Zhu, Shijie Zhou, Jingjing Chen

Research Collection School Of Computing and Information Systems

Vision-Language-Action (VLA) models enable robots to perform manipulation tasks directly from natural language instructions and are increasingly viewed as a foundation for generalist robotic policies. However, their reliability under Out-Of-Distribution (OOD) instructions remains underexplored. In this paper, we reveal a critical failure mode in which VLA policies continue executing visually plausible actions even when the language instruction contradicts the scene. We refer to this phenomenon as linguistic blindness, where VLA policies prioritize visual priors over instruction semantics during action generation. To systematically analyze this issue, we introduce ICBench, a diagnostic benchmark constructed from the LIBERO dataset that probes language–action coupling …

Go to article

Facevalue: Exploring Real-Time Self-View Overlays To Prompt Meaning-Oriented Self-Awareness In Remote Meetings, Gun Woo (Warren) Park, Anthony Tang, Fanny Chevalier Sep 2026

Facevalue: Exploring Real-Time Self-View Overlays To Prompt Meaning-Oriented Self-Awareness In Remote Meetings, Gun Woo (Warren) Park, Anthony Tang, Fanny Chevalier

Research Collection School Of Computing and Information Systems

In remote video meetings, visual non-verbal cues, such as facial expressions or head movements, are seen continuously but often only partially. This increases ambiguity compared to in-person settings and can cause misinterpretation or misalignment between intended and perceived meaning. Motivated by communication theories, we designed FaceValue, a technology probe that augments the self-view with private, real-time overlays. These overlays are subtle, suggestive prompts intended to help attendees reflect on how their cues might be interpreted by others. To invite personal interpretation, FaceValue avoids behavioral labeling and instead aims to support meaning-oriented self-awareness: recognizing when visible cues may unintentionally (mis)communicate intent. …

Go to article

Learning 1-Bit Lidar-Based Localization With Auxiliary Objective, Kaijie Yin, Zhiyuan Zhang, Tian Gao, Wentao Zhu, Cheng-Zhong Xu, Hui Kong Sep 2026

Learning 1-Bit Lidar-Based Localization With Auxiliary Objective, Kaijie Yin, Zhiyuan Zhang, Tian Gao, Wentao Zhu, Cheng-Zhong Xu, Hui Kong

Research Collection School Of Computing and Information Systems

6-DoF LiDAR-based localization is a fundamental capability for autonomous systems operating in large-scale outdoor environments. Many deep-learning-based localization methods have achieved promising performance so far. However, as one of the always-on modules competing for limited on-board computational resources, the localization module is expected to consume only a small portion of the overall compute budget. Most existing learning-based methods are still too heavy for this purpose. In contrast, binary neural networks (BNNs) offer an appealing solution, but the 1-bit compression causes severe information loss and performance drop. In this paper, we address this challenge by proposing Binarized LiDAR-based Localization (BiLoc), the …

Go to article

Neural Symphony Of Flow Experience: Evidence For High-Dimensional Metastable Dynamics, Abdelrahman B. M. Eldaly, Kris Zhangguang Kang, Fiona Fui-Hoon Nah, Leanne Lai-Hang Chan, Keng Siau, Xiao Fan Liu, Richard Huskey, Langtao Chen, Tejaswini Yelamanchili, Rene Weber Sep 2026

Neural Symphony Of Flow Experience: Evidence For High-Dimensional Metastable Dynamics, Abdelrahman B. M. Eldaly, Kris Zhangguang Kang, Fiona Fui-Hoon Nah, Leanne Lai-Hang Chan, Keng Siau, Xiao Fan Liu, Richard Huskey, Langtao Chen, Tejaswini Yelamanchili, Rene Weber

Research Collection School Of Computing and Information Systems

Flow, an optimal experience characterized by deep immersion and engagement in an activity, has been extensively studied in behavioral research. However, its neural dynamic mechanism remains poorly understood. In a within-subject video gaming experiment, we captured neural activity underlying flow, boredom, and anxiety using a 64-channel electroencephalogram (EEG) system. Compared to boredom and anxiety, flow exhibits the highest global functional connectivity, metastability, and dimensionality of dynamic functional connectivity patterns, suggesting that flow is a highly adaptable process that is supported by high-dimensional neural dynamics. Unlike previous studies that focused on identifying static or localized brain activity, we examine the neural …

Go to article

Generalized Logit Adjustment: Improved Fine-Tuning By Mitigating Label Bias In Zero-Shot Vision Models, Beier Zhu, Qianru Sun, Xun Yang, Hanwang Zhang Sep 2026

Generalized Logit Adjustment: Improved Fine-Tuning By Mitigating Label Bias In Zero-Shot Vision Models, Beier Zhu, Qianru Sun, Xun Yang, Hanwang Zhang

Research Collection School Of Computing and Information Systems

Foundation models like CLIP allow zero-shot transfer on various tasks without additional training data. Yet, the zero-shot performance is less competitive than a fully supervised one. Thus, fine-tuning and ensembling are also commonly adopted to better fit the downstream tasks. However, we argue that such prior work has overlooked the inherent biases in foundation models. Due to the highly imbalanced Web-scale training set, foundation models are inevitably skewed toward frequent semantics, and thus the subsequent fine-tuning or ensembling is still biased. In this study, we systematically examine the biases in foundation models and demonstrate the efficacy of our proposed Generalized …

Go to article

Hvi-Cidnet+: Beyond Extreme Darkness For Low-Light Image Enhancement, Kangbiao Shi, Xiaowen Ma, Yixu Feng, Tao Hu, Peng Wu, Guansong Pang, Qingsen Yan Aug 2026

Hvi-Cidnet+: Beyond Extreme Darkness For Low-Light Image Enhancement, Kangbiao Shi, Xiaowen Ma, Yixu Feng, Tao Hu, Peng Wu, Guansong Pang, Qingsen Yan

Research Collection School Of Computing and Information Systems

Low-Light Image Enhancement (LLIE) aims to recover visually pleasing content and details from degraded low-light images. However, existing RGB-based methods often suffer from color bias and brightness artifacts due to inherent high color sensitivity. Although the HSV color space can decouple brightness and color, it introduces noticeable red and black noise artifacts. To address these challenges, we adopt the Horizontal/Vertical-Intensity (HVI) color space for LLIE, which is defined by the HV color map and learnable intensity. The former enforces small distances for red coordinates to alleviate red noise artifacts, while the latter adaptively compresses low-light regions to suppress black noise …

Go to article

Task-Aligned Haze Removal With Semantic-Aware Fusion And Contrast Self-Correction, Jinbin Wang, Aiping Yang, Guosong Jiang, Wenlong Yu, Dongwei Ren, Qinghua Hu Aug 2026

Task-Aligned Haze Removal With Semantic-Aware Fusion And Contrast Self-Correction, Jinbin Wang, Aiping Yang, Guosong Jiang, Wenlong Yu, Dongwei Ren, Qinghua Hu

Research Collection School Of Computing and Information Systems

Adverse haze conditions introduce complex degradations that obscure scene details and distort structural cues critical for object detection, posing persistent challenges for vision‐based sensing systems. Although existing haze removal methods have achieved notable improvements in visual clarity, their optimisation objectives are often misaligned with downstream detection requirements, leading to limited detection performance in real‐world scenarios. To address this issue, this work proposes a task‐aligned weakly supervised haze removal framework, termed Dehaze4Detection, which explicitly aligns low‐level restoration with high‐level detection objectives. The framework incorporates a Semantic‐Aware Multi‐Scale Fusion Module (SMFM) that embeds pixel‐level semantic knowledge into the dehazing process, enabling selective …

Go to article

Activity Transition Graph Generation: How Far Are We?, Jiakun Liu, Peixin Zhang, Han Hu, Yonghui Liu, Wei Minn, Ferdian Thung, Shahar Maoz, Eran Toch, Debin Gao, David Lo Jul 2026

Activity Transition Graph Generation: How Far Are We?, Jiakun Liu, Peixin Zhang, Han Hu, Yonghui Liu, Wei Minn, Ferdian Thung, Shahar Maoz, Eran Toch, Debin Gao, David Lo

Research Collection School Of Computing and Information Systems

Android applications (i.e., apps) are indispensable nowadays and are getting bigger and bigger with an increasing number offunctionalities. To understand how to access functionalities in an app, prior studies proposed tools to model the transitionsbetween functionalities with the activity transition graph (ATG). ATG is an important data structure and has been used forvarious Android app analyses, including app design, understanding, and testing. However, there is no benchmarking work onATG generation. It is still unclear whether the transitions identified by tools are correct and how many transitions are missed.To fill this gap, we manually identified all transitions in 98 applications to …

Go to article

Generation-Augmented Video Corpus Moment Retrieval, Mingjin Kuai, Qianyin Xiao, Juncheng Li, Jin Peng, Lizi Liao, Wei Ji Jul 2026

Generation-Augmented Video Corpus Moment Retrieval, Mingjin Kuai, Qianyin Xiao, Juncheng Li, Jin Peng, Lizi Liao, Wei Ji

Research Collection School Of Computing and Information Systems

Video Corpus Moment Retrieval (VCMR) requires models to efficiently retrieve and precisely locate specific moments relevant to natural language queries within a massive, untrimmed video corpus. However, existing discriminative approaches typically rely on shallow visual-textual feature matching mechanisms, which often struggle to capture fine-grained semantic differences. To address this limitation, we propose Video-GAR, a novel framework that reframes the conventional retrieval task from superficial matching to generative understanding, positing that the capability for query reconstruction evidences deep semantic comprehension. Specifically, Video-GAR orchestrates three synergistic components: To overcome the computational efficiency bottleneck, we construct a Bi-Mamba backbone that leverages the linear …

Go to article

Oscbench: Benchmarking Object State Change In Text-To-Video Generation, Xianjing Han, Bin Zhu, Shiqi Hu, Franklin Mingzhe Li, Patrick Carrington, Roger Zimmermann, Jingjing Chen Jul 2026

Oscbench: Benchmarking Object State Change In Text-To-Video Generation, Xianjing Han, Bin Zhu, Shiqi Hu, Franklin Mingzhe Li, Patrick Carrington, Roger Zimmermann, Jingjing Chen

Research Collection School Of Computing and Information Systems

Text-to-video (T2V) generation models have made rapid progress in producing visually high-quality and temporally coherent videos. However, existing benchmarks primarily focus on perceptual quality, text–video alignment, or physical plausibility, leaving a critical aspect of action understanding largely unexplored: object state change (OSC) explicitly specified in the text prompt. OSC refers to the transformation of an object’s state induced by an action, such as peeling a potato or slicing a lemon. In this paper, we introduce OSCBench, a benchmark specifically designed to assess OSC performance in T2V models. OSCBench is constructed from instructional cooking data and systematically organizes action–object interactions into …

Go to article

Tranx-Adapter: Bridging Artifacts And Semantics Within Mllms For Robust Ai-Generated Image Detection, Wenbin Wang, Yuge Huang, Jianqing Xu, Yue Yu, Jiangtao Yan, Shouhong Ding, Pan Zhou, Yong Luo Jul 2026

Tranx-Adapter: Bridging Artifacts And Semantics Within Mllms For Robust Ai-Generated Image Detection, Wenbin Wang, Yuge Huang, Jianqing Xu, Yue Yu, Jiangtao Yan, Shouhong Ding, Pan Zhou, Yong Luo

Research Collection School Of Computing and Information Systems

Rapid advances in AI-generated image (AIGI) technology enable highly realistic synthesis, threatening public information integrity and security. Recent studies have demonstrated that incorporating texture-level artifact features alongside semantic features into multimodal large language models (MLLMs) can enhance their AIGI detection capability. However, our preliminary analyses reveal that artifact features exhibit high intra-feature similarity, leading to an almost uniform attention map after the softmax operation. This phenomenon causes attention dilution, thereby hindering effective fusion between semantic and artifact features. To overcome this limitation, we propose a lightweight fusion adapter, TranX-Adapter, which integrates a Task-aware Optimal-Transport Fusion that leverages the Jensen-Shannon divergence …

Go to article

Dual-Diffusional Generative Fashion Recommendation, Mingzhe Yu, Lei Wu, Qianru Sun, Yunshan Ma Jul 2026

Dual-Diffusional Generative Fashion Recommendation, Mingzhe Yu, Lei Wu, Qianru Sun, Yunshan Ma

Research Collection School Of Computing and Information Systems

Personalized generative recommender systems have emerged as a promising solution for fashion recommendation. However, existing methods primarily rely on implicit visual embeddings from historical interactions, which often contain preference-irrelevant information and result in insufficient user behavior modeling. Moreover, these models typically generate only item images, providing limited interpretability. To address these limitations, we propose DualFashion, a Dual-Diffusional Generative Fashion Recommendation Architecture that jointly models image and text modalities for personalized and explainable recommendation. DualFashion adopts a dual-diffusion Transformer with image and text branches, where structured attribute-level captions and visual outfit information are jointly used as conditioning signals to model user …

Go to article

Videocreator: An Agentic System For Multi-Turn Video Production, Zhengyang Liang, Yan Shu, Cathal Gurrin, Nicu Sebe, Lizi Liao Jun 2026

Videocreator: An Agentic System For Multi-Turn Video Production, Zhengyang Liang, Yan Shu, Cathal Gurrin, Nicu Sebe, Lizi Liao

Research Collection School Of Computing and Information Systems

Recent advances in video generation models enable visually compelling single clips. However, real-world video creation is inherently continuous and iterative: creators refine content over multiple rounds while maintaining narrative, style, and entity consistency. Existing standalone generators are largely stateless and lack memory of previously generated segments, making it difficult to produce a coherent and consistent video project. To address this gap, we present VideoCreator, a unified video agent that integrates generation and understanding with a project-level memory system. VideoCreator leverages understanding capabilities to perform fine-grained analysis of newly produced content and uses persistent memory to retain and reuse prior context …

Go to article

Enhancing Pointing Gestures Of Non-Hmd Users In Asymmetric Collocated Mixed Reality Collaboration, Nam-Dang Vo, Van-Vinh Thai, Anthony Tang, Khanh-Duy Le Jun 2026

Enhancing Pointing Gestures Of Non-Hmd Users In Asymmetric Collocated Mixed Reality Collaboration, Nam-Dang Vo, Van-Vinh Thai, Anthony Tang, Khanh-Duy Le

Research Collection School Of Computing and Information Systems

A common collocated group setting in mixed-reality (MR) collaboration is a person wearing a MR headset (HMD user) and presenting MR contents to audiences who are not provided with such specialized devices (Non-HMD users). In this setting, while Non-HMD users can view the MR environment shown on a large physical display, it still remains challenging for the HMD user to interpret their pointing gesture when they spatially refer to objects in the MR environment. To address this, we designed and evaluated two pointing techniques—SCREEN and SCREEN+SPACE—that support Non-HMD users in referring to MR content. Screen pointing allows users to refer …

Go to article

“From Remembering To Shaping”: Narrating Shared Experiences By Co-Designing Cultural Heritage Artifacts In Collaborative Vr, Yushang Yang, Fanxu Meng, Fiona Fui-Hoon Nah, L. C. Ray Jun 2026

“From Remembering To Shaping”: Narrating Shared Experiences By Co-Designing Cultural Heritage Artifacts In Collaborative Vr, Yushang Yang, Fanxu Meng, Fiona Fui-Hoon Nah, L. C. Ray

Research Collection School Of Computing and Information Systems

The ways people remember and recall places reveal an invisible aspect of cultural heritage (CH), reflecting how individuals and communities relate to these places. Heritage is communal, emerging through collaboratively constructed narratives rather than individual records. To probe how people may share collective memories, we designed an immersive two-person workflow for collaboratively co-designing 3D artifacts and environments in virtual heritage locations, using Generative AI (GenAI) to instantiate these intangible memories. Observations of the co-creation process revealed that participants merged prompts and model placements when negotiating different perspectives. They used spatial operations to compose scenes, and also to express personal and …

Go to article

Not Too Early, Not All At Once: Design Tensions In Ai-Mediated Self-Disclosure In Online Dating, Pei-Hua Tsai, Tianyi Zhang, Emran Bin Elias Poh, Anthony Tang, Yung-Ju Chang Jun 2026

Not Too Early, Not All At Once: Design Tensions In Ai-Mediated Self-Disclosure In Online Dating, Pei-Hua Tsai, Tianyi Zhang, Emran Bin Elias Poh, Anthony Tang, Yung-Ju Chang

Research Collection School Of Computing and Information Systems

Online dating relies on self-disclosure, yet initial conversations are fragile: users must navigate uncertainty around timing, boundaries, and reciprocity with little shared context. While advances in AI raise the possibility of mediating disclosure, how such support might reshape the experience of early-stage relational disclosure remains underexplored. We conducted 29 semi-structured interviews to examine how daters envision AI-mediated self-disclosure in online dating. Our findings surface recurring design tensions rather than simple opportunities or risks. Participants welcomed guidance that could pace disclosure, support reflection, and reduce social awkwardness, but stressed preserving agency and authorship. They valued interpretive assistance for sense-making of ambiguous …

Go to article

Happycal: Designing Text And Image-Based Supports For Savouring Positive Work Experiences, Molly Stewart, Minghao Cai, Anthony Tang, Sam Liu, Chris Mosunic, Sowmya Somanath Jun 2026

Happycal: Designing Text And Image-Based Supports For Savouring Positive Work Experiences, Molly Stewart, Minghao Cai, Anthony Tang, Sam Liu, Chris Mosunic, Sowmya Somanath

Research Collection School Of Computing and Information Systems

Savouring positive work experiences can promote positive affect and well-being at work, yet there is limited guidance on how digital applications can support workers to engage in savouring. We developed HappyCal, a work-focused savouring application offering two forms of savouring support: text-based, a common modality in workplace reflection tools, and images, a largely unexplored approach in work-related savouring. We conducted an exploratory qualitative study where participants (N=36) used HappyCal over five days and engaged in savouring through either a text-only modality (n=17) or text input paired with image output (n=19). We found that (1) participants in both groups reported heightened …

Go to article

A Pruning-Based Question-Answering For Interactive Video Search: A Simple Baseline, Yu Tong Cheng, Phuong Anh Nguyen, Chong-Wah Ngo Jun 2026

A Pruning-Based Question-Answering For Interactive Video Search: A Simple Baseline, Yu Tong Cheng, Phuong Anh Nguyen, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

There are various factors affecting the performance of video search. An imprecise query will enlarge search space and reduce the discriminative power of ranking functions. This problem is further exacerbated by the presence of numerous visually or semantically similar videos in large datasets. Consequently, users need to painstakingly browse through many highly similar candidates to locate the search target, leading to increased cognitive load and inefficient searching. Ideally, engaging users through interactive questioning to resolve uncertainties in the search process is an effective strategy for progressively narrowing down the search space. However, despite rapid advances in deep learning, generating informative …

Go to article

Sam3-Litetext: An Anatomical Study Of The Sam3 Text Encoder For Efficient Vision-Language Segmentation, Chengxi Zeng, Yuxuan Jiang, Ge Gao, Shuai Wang, Duolikun Danier, Bin Zhu, Stevan Rudinac, David Bull, Fan Zhang Jun 2026

Sam3-Litetext: An Anatomical Study Of The Sam3 Text Encoder For Efficient Vision-Language Segmentation, Chengxi Zeng, Yuxuan Jiang, Ge Gao, Shuai Wang, Duolikun Danier, Bin Zhu, Stevan Rudinac, David Bull, Fan Zhang

Research Collection School Of Computing and Information Systems

Vision-language segmentation models such as SAM3 enable flexible, prompt-driven visual grounding, but inherit large, general-purpose text encoders originally designed for open-ended language understanding. In practice, segmentation prompts are short, structured, and semantically constrained, leading to substantial over-provisioning in text encoder capacity and persistent computational and memory overhead. In this paper, we perform a large-scale anatomical analysis of text prompting in vision–language segmentation, covering 404,796 real prompts across multiple benchmarks. Our analysis reveals severe redundancy: most context windows are underutilized, vocabulary usage is highly sparse, and text embeddings lie on a low-dimensional manifold despite high-dimensional representations. Motivated by these findings, we …

Go to article

Co-Designing With Autistic Livestreamers: Care, Constraints, And Trade-Offs In Livestreaming, Terrance Mok, Anthony Tang, Lora Oehlberg Jun 2026

Co-Designing With Autistic Livestreamers: Care, Constraints, And Trade-Offs In Livestreaming, Terrance Mok, Anthony Tang, Lora Oehlberg

Research Collection School Of Computing and Information Systems

Autistic livestreamers use platforms like Twitch for social connection, self-expression, and community, but these spaces also impose ongoing social and emotional demands. Prior work has documented these experiences, but less is known about what autistic creators themselves envision for the tools and platforms they use. We address this gap through a Research through Design (RtD) co-design study with three autistic Twitch streamers, using speculative artefacts as discussion prompts to explore how participants reasoned about potential livestreaming technologies. Across three co-design activities, we identify three overarching tensions shaping autistic streaming practice: Expression versus Misinterpretation and Harm; Public Participation versus Control and …

Go to article

History To Future: Evolving Agent With Experience And Thought For Zero-Shot Vision-And-Language Navigation, Guangzhao Dai, Shuo Wang, Zihan Wang, Guo-Sen Xie, Yang Yang, Jinshan Pan, Qianru Sun, Xiangbo Shu Jun 2026

History To Future: Evolving Agent With Experience And Thought For Zero-Shot Vision-And-Language Navigation, Guangzhao Dai, Shuo Wang, Zihan Wang, Guo-Sen Xie, Yang Yang, Jinshan Pan, Qianru Sun, Xiangbo Shu

Research Collection School Of Computing and Information Systems

Vision-and-Language Navigation in Continuous Environment (VLN-CE) requires an agent to follow language instructions to navigate the target destination. With the advancement of large language models (LLMs), recent efforts have explored adapting them for zero-shot VLN-CE, offering a promising solution in addressing the drawbacks of poor generalization in the training-based paradigm. However, existing LLM-based works primarily perform naive reasoning for decision-making and lack feedback, e.g., reviewing historical errors and predicting future potentials. Consequently, it may suffer from continuous failure for those initial error tasks. In this paper, we rethink LLM-based zero-shot VLN-CE and propose a new paradigm, named EvoNav, to improve …

Go to article

Super Lidar Intensity For Robotic Perception, Wei Gao, Jie Zhang, Mingle Zhao, Zhiyuan Zhang, Shu Kong, Maani Ghaffari, Dezhen Song, Chengzhong Xu, Hui Kong Apr 2026

Super Lidar Intensity For Robotic Perception, Wei Gao, Jie Zhang, Mingle Zhao, Zhiyuan Zhang, Shu Kong, Maani Ghaffari, Dezhen Song, Chengzhong Xu, Hui Kong

Research Collection School Of Computing and Information Systems

Conventionally, human intuition defines vision as a modality of passive optical sensing, relying on ambient light to perceive the environment. However, active optical sensing, which involves emitting and receiving signals, offers unique advantages by capturing both radiometric and geometric properties of the environment, independent of external illumination conditions. This work focuses on advancing active optical sensing using Light Detection and Ranging (LiDAR), which captures intensity data, enabling the estimation of surface reflectance that remains invariant under varying illumination. Such properties are crucial for robotic perception tasks, including detection, recognition, segmentation, and Simultaneous Localization and Mapping (SLAM). A key challenge with …

Go to article

Learning Feature Inversion For Multi-Class Anomaly Detection Under General-Purpose Coco-Ad Benchmark, Jiangning Zhang, Chengjie Wang, Xiangtai Li, Guanzhong Tian, Zhucun Xue, Yong Liu, Guansong Pang, Dacheng Tao Apr 2026

Learning Feature Inversion For Multi-Class Anomaly Detection Under General-Purpose Coco-Ad Benchmark, Jiangning Zhang, Chengjie Wang, Xiangtai Li, Guanzhong Tian, Zhucun Xue, Yong Liu, Guansong Pang, Dacheng Tao

Research Collection School Of Computing and Information Systems

Anomaly detection (AD) is often focused on detecting anomaly areas for industrial quality inspection and medical lesion examination. However, due to the specific scenario targets, the data scale for AD is relatively small, and evaluation metrics are still deficient compared to classic vision tasks, such as object detection and semantic segmentation. To fill these gaps, this work first constructs a large-scale and general-purpose COCO-AD dataset by extending COCO to the AD field. This enables fair evaluation and sustainable development for different methods on this challenging benchmark. Moreover, current metrics such as AU-ROC have nearly reached saturation on simple datasets, which …

Go to article

From Spatial To Actions: Grounding Vision-Language-Action Model In Spatial Foundation Priors, Zhengshen Zhang, Hao Li, Yalun Dai, Zhengbang Zhu, Lei Zhou, Chenchen Liu, Dong Wang, Francis E. H. Tay, Sijin Chen, Ziwei Liu, Yuxiao Liu, Xinghang Li, Pan Zhou Apr 2026

From Spatial To Actions: Grounding Vision-Language-Action Model In Spatial Foundation Priors, Zhengshen Zhang, Hao Li, Yalun Dai, Zhengbang Zhu, Lei Zhou, Chenchen Liu, Dong Wang, Francis E. H. Tay, Sijin Chen, Ziwei Liu, Yuxiao Liu, Xinghang Li, Pan Zhou

Research Collection School Of Computing and Information Systems

Existing vision-language-action (VLA) models act in 3D real-world but are typically built on 2D encoders, leaving a spatial reasoning gap that limits generalization and adaptability. Recent 3D integration techniques for VLAs either require specialized sensors and transfer poorly across modalities, or inject weak cues that lack geometry and degrade vision-language alignment. In this work, we introduce FALCON (From Spatial to Action), a novel paradigm that injects rich 3D spatial tokens into the action head. FALCON leverages spatial foundation models to deliver strong geometric priors from RGB alone, and includes an Embodied Spatial Model that can optionally fuse depth, or pose …

Go to article

Semat: Semantic Enhanced Natural Image Interactive Matting, Ruihao Xia, Yu Liang, Peng-Tao Jiang, Hao Zhang, Qianru Sun, Yang Tang, Bo Li, Pan Zhou Apr 2026

Semat: Semantic Enhanced Natural Image Interactive Matting, Ruihao Xia, Yu Liang, Peng-Tao Jiang, Hao Zhang, Qianru Sun, Yang Tang, Bo Li, Pan Zhou

Research Collection School Of Computing and Information Systems

Recent approaches attempt to adapt powerful interactive segmentation models, such as SAM, to interactive matting and fine-tune the models based on synthetic matting datasets. However, models trained on synthetic data fail to generalize to complex and occlusion scenes. We address this challenge by proposing a new matting dataset based on the COCO dataset, namely COCO-Matting. It selects real-world complex images from COCO and converts semantic segmentation masks to matting labels. The built COCO-Matting comprises an extensive collection of 36,980 human instance-level alpha mattes in complex natural scenarios. Furthermore, existing SAM-based matting methods extract intermediate features and masks from a frozen …

Go to article

Next Last