Computer Sciences Commons^™

Modfinity: Unsupervised Domain Adaptation With Multimodal Information Flow Intertwining, Shanglin Liu, Jianming Lv, Jingdan Kang, Huaidong Zhang, Zequan Liang, Shengfeng He

Research Collection School Of Computing and Information Systems

Multimodal unsupervised domain adaptation leverages unlabeled data in the target domain to enhance multimodal systems continuously. While current state-of-the-art methods encourage interaction between sub-models of different modalities through pseudo-labeling and feature-level exchange, varying sample quality across modalities can lead to the propagation of inaccurate information, resulting in error accumulation. To address this, we propose Modal-Affinity Multimodal Domain Adaptation (MODfinity), a method that dynamically manages multimodal information flow through fine-grained control over teacher model selection, guiding information intertwining at both feature and label levels. By treating labels as an independent modality, MODfinity enables balanced performance assessment across modalities, employing a novel …

A Multimodal Fusion Model Leveraging Mlp Mixer And Handcrafted Features-Based Deep Learning Networks For Facial Palsy Detection, Heng Yim Nicole Oo, Min Hun Lee, Jeong Hoon Lim

Research Collection School Of Computing and Information Systems

Algorithmic detection of facial palsy offers the potential to improve current practices, which usually involve labor-intensive and subjective assessments by clinicians. In this paper, we present a multimodal fusion-based deep learning model that utilizes an MLP mixer-based model to process unstructured data (i.e. RGB images or images with facial line segments) and a feed-forward neural network to process structured data (i.e. facial landmark coordinates, features of facial expressions, or handcrafted features) for detecting facial palsy. We then contribute to a study to analyze the effect of different data modalities and the benefits of a multimodal fusion-based approach using videos of …

Ntire 2025 Challenge On Event-Based Image Deblurring: Methods And Results, Lei Sun, Et. Al.

Research Collection School Of Computing and Information Systems

This paper presents an overview of NTIRE 2025, the First Challenge on Event-Based Image Deblurring, detailing the proposed methodologies and corresponding results. The primary goal of the challenge is to design an event-based method that achieves high-quality image deblurring, with performance quantitatively assessed using Peak Signal-toNoise Ratio (PSNR). Notably, there are no restrictions on computational complexity or model size. The task focuses on leveraging both events and images as inputs for singleimage deblurring. A total of 199 participants registered, among whom 15 teams successfully submitted valid results, offering valuable insights into the current state of eventbased image deblurring. We anticipate …

Event-Based Eye Tracking: Event-Based Vision Workshop 2025, Qinyu Chen, Et. Al.

Research Collection School Of Computing and Information Systems

No abstract provided.

Human-Computer Interaction And Artificial Intelligence For Ageing Population, Keng Siau, Hailiang Wang, Fiona Fui-Hoon Nah, Runyu Wang, Ruitong Che, Can Liu

Research Collection School Of Computing and Information Systems

As the global population ages rapidly, the field of human-computer interaction (HCI) is in urgent need of innovation, redesign, and reengineering to meet the evolving needs of older adults. The older demographic faces a range of challenges—including physical limitations, cognitive decline, reduced social in-tegration, and varying levels of technological literacy—that can hinder effective engagement with digital technologies. In response to these challenges, research-ers and designers are using inclusive and adaptive approaches to enhance acces-sibility, usability, and emotional well-being. This paper reviews key design prin-ciples in HCI for the ageing population and discusses how artificial intelligence (AI) tools, such as voice …

Unlocking The Power Of Socio-Knowledge Association For Enterprise Risk Identification In Stock Market, Zhenghao Liu, Keng Siau, Shaochen Yang, Feicheng Ma

Research Collection School Of Computing and Information Systems

Potential risk signals reflected in supply chain and equity connections between enterprises and social connections between investors are becoming crucial to identifying enterprise risks in addition to basic financial indicators. Traditional risk management systems face challenges in adapting to these complexities, highlighting the need for a proactive paradigm shift in risk management. Leveraging graph models such as social networks and knowledge graphs offers a promising approach to identifying and managing potential associated risks effectively. To bridge existing research gaps, a novel risk identification framework driven by social-knowledge graphs has been proposed, integrating graph deep learning and reinforcement learning techniques guided …

Efficient And Green Large Language Models For Software Engineering: Literature Review, Vision, And The Road Ahead, Jieke Shi, Zhou Yang, David Lo

Research Collection School Of Computing and Information Systems

Large Language Models (LLMs) have recently shown remarkable capabilities in various software engineering tasks, spurring the rapid growth of the Large Language Models for Software Engineering (LLM4SE) area. However, limited attention has been paid to developing efficient LLM4SE techniques that demand minimal computational cost, time, and memory resources, as well as green LLM4SE solutions that reduce energy consumption, water usage, and carbon emissions. This article aims to redirect the focus of the research community toward the efficiency and greenness of LLM4SE, while also sharing potential research directions to achieve this goal. It commences with a brief overview of the significance …

Deepvec: State-Vector Aware Test Case Selection For Enhancing Recurrent Neural Network, Zhonghao Jiang, Meng Yan, Li Huang, Weifeng Sun, Chao Liu, Song Sun, David Lo

Research Collection School Of Computing and Information Systems

Deep Neural Networks (DNN) have realized significant achievements across various application domains. There is no doubt that testing and enhancing a pre-trained DNN that has been deployed in an application scenario is crucial, because it can reduce the failures of the DNN. DNN-driven software testing and enhancement require large amounts of labeled data. The high cost and inefficiency caused by the large volume of data of manual labeling, and the time consumption of testing all cases in real scenarios are unacceptable. Therefore, test case selection technologies are proposed to reduce the time cost by selecting and only labeling representative test …

On Lexicographic Proof Rules For Probabilistic Termination, Krishnendu Chatterjee, Ehsan Kafshdar Goharshady, Petr Novotný, Jiří Zárevucký, Dorde Zikelic

Research Collection School Of Computing and Information Systems

We consider the almost-sure (a.s.) termination problem for probabilistic programs, which are a stochastic extension of classical imperative programs. Lexicographic ranking functions provide a sound and practical approach for termination of non-probabilistic programs, and their extension to probabilistic programs is achieved via lexicographic ranking supermartingales (LexRSMs). However, LexRSMs introduced in the previous work have a limitation that impedes their automation: all of their components have to be non-negative in all reachable states. This might result in a LexRSM not existing even for simple terminating programs. Our contributions are twofold. First, we introduce a generalization of LexRSMs that allows for some …

Building Narratives And Probing Concepts: Preparing Materials For Co-Design With Autistic Livestreamers, Terrance Mok, Tyson Hartley, Anthony Tang, Adam Mccrimmon, Lora Oehlberg

Research Collection School Of Computing and Information Systems

Based on ten semi-structured interviews with autistic Twitch streamers, we introduce a series of scenario-based design narratives coupled with technology design concepts as a starting point for co-design discussion about autistic streaming. This work builds on prior thematic analysis of the unique intersection between autism and livestreaming. Our user-centered scenarios highlight the needs, goals, and challenges of autistic individuals in livestreaming contexts. By using evocative narratives, the scenarios serve to facilitate empathy and deeper engagement with the needs of autistic users, and help facilitate and support co-creative dialogues and discussions about new technology designs. We contribute this starting point for …

Meta-Learning Hyperparameters For Foundation Model Adaptation In Remote-Sensing Imagery, Zichen Tian, Yaoyao Liu, Qianru Sun

Research Collection School Of Computing and Information Systems

Training large foundation models of remote-sensing (RS) images is almost impossible due to the limited and long-tailed data problems. Fine-tuning natural image pre-trained models on RS images is a straightforward solution. To reduce computational costs and improve performance on tail classes, existing methods apply parameter-efficient fine-tuning (PEFT) techniques, such as LoRA and AdaptFormer. However, we observe that fixed hyperparameters -- such as intra-layer positions, layer depth, and scaling factors, can considerably hinder PEFT performance, as fine-tuning on RS images proves highly sensitive to these settings. To address this, we propose MetaPEFT, a method incorporating adaptive scalers that dynamically adjust module …

Generative Artificial Intelligence Dependency: Scale Development, Validation, And Its Motivational, Behavioral, And Psychological Correlates, Adalia Yin Hui Goh

Dissertations and Theses Collection (Open Access)

The growing integration of generative artificial intelligence (AI) into everyday life has raised questions about its potential psychological and behavioral consequences. The present research develops and validates the Generative AI Dependency Scale, a multidimensional tool developed to assess individual differences in dependency on generative AI systems. Across six studies involving 1,223 participants from the United States and Singapore, the Generative AI Dependency Scale demonstrated strong psychometric properties, including a stable three-factor structure (cognitive preoccupation, negative consequences, withdrawal) and good test-retest reliability (ICC = .85). Confirmatory factor analysis supported a higher-order dependency construct, and scalar measurement invariance was established across sex …

Diversity-Augmented Training For Generalizable Ai Agents, Wenjun Li

Dissertations and Theses Collection (Open Access)

Deep Reinforcement Learning (RL) has achieved remarkable success over the past decade, from superhuman performance in video games to real-world applications like robotics. However, RL models often lack generalization, making them unreliable when deployed in unfamiliar scenarios. For example, robots must adapt to varying terrains with different slopes and obstacles, yet standard RL training does not explicitly promote such adaptability. While various methods have been proposed to enhance RL robustness, achieving reliable generalization remains an open challenge.

This dissertation focuses on improving the generalization capability of agents in three major settings: infinite horizon RL agents, finite horizon RL agents, and …

Evaluation Of Pre-Trained Vision Language Models In Challenging Contexts, Kankan Zhou

Dissertations and Theses Collection (Open Access)

The rapid advancement and proliferation of pre-trained vision-language models (VLMs) have heralded a new era in the realm of artificial intelligence (AI), opening up unprecedented opportunities and challenges alike. This dissertation sets forth on an ambitious and comprehensive journey to critically evaluate the performance and limitations of pre-trained VLMs, particularly in complex and challenging contexts that test the bounds of their capabilities. Our focus is twofold: to rigorously assess the extent of bias embedded in these models, and to meticulously scrutinize their reasoning abilities, highlighting parallels and disparities between machine and human cognition.

We initiate our exploration with a targeted …

Enhancing Sub-Optimal Trajectory Stitching: Spatial Composition Rvs For Offline Rl, Sheng Zang, Zhiguang Cao, Bo An, Senthilnath Jayavelu, Xiaoli Li

Research Collection School Of Computing and Information Systems

Reinforcement learning via supervised learning (RvS) has been known as a burgeoning paradigm for offline reinforcement learning (RL). While return-conditioned RvS (RvS-R) predominates across a wide range of datasets pertaining to the offline RL tasks, recent findings suggest that goal-conditioned RvS (RvS-G) outperforms in specific sub-optimal datasets where trajectory stitching is crucial for achieving optimal performance. However, the underlying reasons for this superiority remain insufficiently explored. In this paper, employing didactic experiments and theoretical analysis, we reveal that the proficiency of RvS-G in stitching trajectories arises from its adeptness in generalizing to unknown goals during evaluation. Building on this insight, …

Worldcuisines: A Massive-Scale Benchmark For Multilingual And Multicultural Visual Question Answering On Global Cuisines, Genta Indra Winata, Et. Al

Research Collection School Of Computing and Information Systems

Vision Language Models (VLMs) often struggle with culture-specific knowledge, particularly in languages other than English and in underrepresented cultural contexts. To evaluate their understanding of such knowledge, we introduce WorldCuisines, a massive-scale benchmark for multilingual and multicultural, visually grounded language understanding. This benchmark includes a visual question answering (VQA) dataset with text-image pairs across 30 languages and dialects, spanning 9 language families and featuring over 1 million data points, making it the largest multicultural VQA benchmark to date. It includes tasks for identifying dish names and their origins. We provide evaluation datasets in two sizes (12k and 60k instances) alongside …

Fixdrive: Automatically Repairing Autonomous Vehicle Driving Behaviour For $0.08 Per Violation, Yang Sun, Christopher M. Poskitt, Kun Wang, Jun Sun

Research Collection School Of Computing and Information Systems

Autonomous Vehicles (AVs) are advancing rapidly, with Level-4 AVs already operating in real-world conditions. Current AVs, however, still lag behind human drivers in adaptability and performance, often exhibiting overly conservative behaviours and occasionally violating traffic laws. Existing solutions, such as runtime enforcement, mitigate this by automatically repairing the AV's planned trajectory at runtime, but such approaches lack transparency and should be a measure of last resort. It would be preferable for AV repairs to generalise beyond specific incidents and to be interpretable for users. In this work, we propose FixDrive, a framework that analyses driving records from near-misses or law …

Decictor: Towards Evaluating The Robustness Of Decision-Making In Autonomous Driving Systems, Mingfei Cheng, Xiaofei Xie, Yuan Zhou, Junjie Wang, Guozhu Meng, Kairui Yang

Research Collection School Of Computing and Information Systems

Autonomous Driving System (ADS) testing is crucial in ADS development, with the current primary focus being on safety. However, the evaluation of non-safety-critical performance, particularly the ADS's ability to make optimal decisions and produce optimal paths for autonomous vehicles (AVs), is also vital to ensure the intelligence and reduce risks of AVs. Currently, there is little work dedicated to assessing the robustness of ADSs' path-planning decisions (PPDs), i.e., whether an ADS can maintain the optimal PPD after an insignificant change in the environment. The key challenges include the lack of clear oracles for assessing PPD optimality and the difficulty in …

Hello Again! Llm-Powered Personalized Agent For Long-Term Dialogue, Hao Li, Chenghao Yang, An Zhang, Yang Deng, Xiang Wang, Tat-Seng Chua

Research Collection School Of Computing and Information Systems

Open-domain dialogue systems have seen remarkable advancements with the development of large language models (LLMs). Nonetheless, most existing dialogue systems predominantly focus on brief single-session interactions, neglecting the real-world demands for long-term companionship and personalized interactions with chatbots. Crucial to addressing this real-world need are event summary and persona management, which enable reasoning for appropriate long-term dialogue responses. Recent progress in the human-like cognitive and reasoning capabilities of LLMs suggests that LLM-based agents could significantly enhance automated perception, decision-making, and problem-solving. In response to this potential, we introduce a model-agnostic framework, the Long-term Dialogue Agent (LD-Agent), which incorporates three independently …

Towards Reliable Ml: Data Attribution And Adversarial Robustness, Xiaosen Zheng

Dissertations and Theses Collection (Open Access)

Modern machine learning (ML) models achieve remarkable success, but face critical reliability challenges. This thesis advances two pillars of reliable ML systems: interpretability through data attribution and robustness against adversarial threats.

In the first part, we develop novel data attribution methods to elucidate the data-model relationship. We establish the critical role of memorization in model generalization through token-level influence analysis, extend sample-level attribution to diffusion models with effective approximation techniques, and introduce REGMIX, a group-level approach that predicts data mixture performance using small-scale experiments. These contributions provide practitioners with scalable tools to audit training data impacts across modalities.

The second …

Disentangling User Preferences Towards Self-Interpretable Recommender Systems, Nhu Thuat Tran

Dissertations and Theses Collection (Open Access)

Understanding user preferences remains a central challenge in recommender systems due to their inherently complex, unstructured, and multi-faceted nature, exacerbated by the sparsity of user interaction data. Traditional approaches often compress user interests into a single latent vector, overlooking the fact that user preferences are typically shaped by multiple underlying factors that differ across individuals. These latent drivers are not directly observable and must be discovered through unsupervised modeling, further complicated by limited historical interactions per user.

This dissertation addresses these challenges by introducing a principled framework for multiinterest modeling, which disentangles user behaviors into multiple latent factors to better …

Tailoring Transformer-Based Deep Learning For Code Generation And Translation, Imam Nur Bani Yusuf

Dissertations and Theses Collection (Open Access)

Software is increasingly pervasive in modern society, making the effective translation of human intent into code essential. Novice programmers often struggle with domain-specific code due to limited background knowledge, while experienced developers face challenges in maintaining evolving largescale codebases. Traditional pattern-based approaches address these issues, but such approaches are task-specific and require significant adaptation for different tasks. Transformer-based models offer a more flexible alternative, as the same architecture can be tailored for diverse programming tasks.

This dissertation investigates how Transformer-based models can be customized for various code generation and translation tasks. First, it introduces Transformer-based approaches that assist end-users with …

Oscar: Object Status And Contextual Awareness For Recipes To Support Non-Visual Cooking, Franklin Mingzhe Li, Kaitlyn Ng, Bin Zhu, Patrick Carrington

Research Collection School Of Computing and Information Systems

Following recipes while cooking is an important but difficult task for visually impaired individuals. We developed OSCAR (Object Status Context Awareness for Recipes), a novel approach that provides recipe progress tracking and context-aware feedback on the completion of cooking tasks through tracking object statuses. OSCAR leverages both Large-Language Models (LLMs) and Vision-Language Models (VLMs) to manipulate recipe steps, extract object status information, align visual frames with object status, and provide cooking progress tracking log. We evaluated OSCAR’s recipe following functionality using 173 YouTube cooking videos and 12 real-world non-visual cooking videos to demonstrate OSCAR’s capability to track cooking steps and …

Seaexam And Seabench: Benchmarking Llms With Local Multilingual Questions In Southeast Asia, Chaoqun Liu, Wenxuan Zhang, Jiahao Ying, Mahani Aljunied, Anh Tuan Luu, Lidong Bing

Research Collection School Of Computing and Information Systems

This study introduces two novel benchmarks, SeaExam and SeaBench, designed to evalu ate the capabilities of Large Language Models (LLMs) in Southeast Asian (SEA) application scenarios. Unlike existing multilingual datasets primarily derived from English translations, these benchmarks are constructed based on real world scenarios from SEA regions. SeaExam draws from regional educational exams to form a comprehensive dataset that encompasses sub jects such as local history and literature. In contrast, SeaBench is crafted around multi turn, open-ended tasks that reflect daily inter actions within SEA communities. Our evalua tions demonstrate that SeaExam and SeaBench more effectively discern LLM performance on …

Intention Is All You Need: Refining Your Code From Your Intention, Qi Guo, Xiaofei Xie, Shangqing Liu, Ming Hu, Xiaohong Li, Lei Bu

Research Collection School Of Computing and Information Systems

Code refinement aims to enhance existing code by addressing issues, refactoring, and optimizing to improve quality and meet specific requirements. As software projects scale in size and complexity, the traditional iterative exchange between reviewers and developers becomes increasingly burdensome. While recent deep learning techniques have been explored to accelerate this process, their performance remains limited, primarily due to challenges in accurately understanding reviewers’ intents. This paper proposes an intention-based code refinement technique that enhances the conventional comment-to-code process by explicitly extracting reviewer intentions from the comments. Our approach consists of two key phases: Intention Extraction and Intention Guided Revision Generation. …

Tensorjsfuzz: Effective Testing Of Web-Based Deep Learning Frameworks Via Input-Constraint Extraction, Lili Quan, Xiaofei Xie, Qianyu Guo, Lingxiao Jiang, Sen Chen, Junjie Wang, Xiaohong Li

Research Collection School Of Computing and Information Systems

The 2025 ACM Web Conference (WWW '25) took place from April 28 to May 2, 2025, in the Sydney Convention & Exhibition Centre, Australia. Its logo, featuring the Sydney Harbour Bridge, symbolizes the core "connecting" function of the Web. Formerly known as the International World Wide Web Conference (WWW), this event originated at CERN in 1994 and has long served as the premier venue for presenting and discussing research, development, standards, and applications related to the Web.The 2025 ACM Web Conference (WWW'25) took place from April 28 to May 2, 2025, in the Sydney Convention & Exhibition Centre, Australia. Its …

Rotation-Adaptive Point Cloud Domain Generalization Via Intricate Orientation Learning, Bangzhen Liu, Chenxi Zheng, Xuemiao Xu, Cheng Xu, Huaidong Zhang, Shengfeng He

Research Collection School Of Computing and Information Systems

The vulnerability of 3D point cloud analysis to unpredictable rotations poses an open yet challenging problem: orientation-aware 3D domain generalization. Cross-domain robustness and adaptability of 3D representations are crucial but not easily achieved through rotation augmentation. Motivated by the inherent advantages of intricate orientations in enhancing generalizability, we propose an innovative rotation-adaptive domain generalization framework for 3D point cloud analysis. Our approach aims to alleviate orientational shifts by leveraging intricate samples in an iterative learning process. Specifically, we identify the most challenging rotation for each point cloud and construct an intricate orientation set by optimizing intricate orientations. Subsequently, we employ …

“I Can Run At Night!”: Using Augmented Reality To Support Nighttime Guided Running For Low-Vision Runners, Yuki Abe, Keisuke Matsushima, Kotaro Hara, Daisuke Sakamoto, Tetsuo Ono

Research Collection School Of Computing and Information Systems

Dark environment challenges low-vision (LV) individuals to engage in running by following sighted guide—a Caller-style guided running—due to insufficient illumination, because it prevents them from using their residual vision to follow the guide and be aware about their environment. We design, develop, and evaluate RunSight, an augmented reality (AR)-based assistive tool to support LV individuals to run at night. RunSight combines see-through HMD and image processing to enhance one’s visual awareness of the surrounding environment (e.g., potential hazard) and visualize the guide’s position with AR-based visualization. To demonstrate RunSight’s efficacy, we conducted a user study with 8 LV runners. The …

Cyberoception: Finding A Painlessly-Measurable New Sense In The Cyberworld Towards Emotion-Awareness In Computing, Tadashi Okoshi, Zexiong Gao, Yi Zhen Tan, Takumi Karasawa, Takeshi Miki, Wataru Sasaki, Rajesh Krishna Balan

Research Collection School Of Computing and Information Systems

In Affective computing, recognizing users’ emotions accurately is the basis of affective human–computer interaction. Understanding users’ interoception contributes to a better understanding of individually different emotional abilities, which is essential for achieving inter-individually accurate emotion estimation. However, existing interoception measurement methods, such as the heart rate discrimination task, have several limitations, including their dependence on a well-controlled laboratory environment and precision apparatus, making monitoring users’ interoception challenging. This study aims to determine other forms of data that can explain users’ interoceptive or similar states in their real-world lives and propose a novel hypothetical concept “cyberoception,” a new sense (1) which …

Mosmac: A Multi-Agent Reinforcement Learning Benchmark On Sequential Multi-Objective Tasks, Minghong Geng, Shubham Pateria, Budhitama Subagdja, Ah-Hwee Tan

Research Collection School Of Computing and Information Systems

Recent advancements in multi-agent reinforcement learning (MARL) have demonstrated success on various cooperative multi-agent tasks. However, current benchmarks often fall short of representing realistic scenarios that demand agents to execute sequential tasks over long temporal horizons while balancing multiple objectives. To address this limitation, we introduce multi-objective SMAC (MOSMAC), a comprehensive MARL benchmark designed to evaluate MARL methods on tasks involving multiple objectives, sequential subtask assignments, and varying temporal horizons. MOSMAC requires agents to tackle a series of interconnected subtasks in StarCraft II while simultaneously optimizing for multiple objectives, including combat, safety, and navigation. Through rigorous evaluation of nine state-of-the-art …