Computer Sciences Commons^™

Unified Generative And Discriminative Training For Multi-Modal Large Language Models, Wei Chow, Juncheng Li, Kaihang Pan, Qifan Yu, Hao Fei, Zhiqi Ge, Shuai Yang, Siliang Teng, Hanwang Zhang, Qianru Sun

Research Collection School Of Computing and Information Systems

In recent times, Vision-Language Models (VLMs) have been trained under two predominant paradigms. Generative training has enabled Multimodal Large Language Models (MLLMs) to tackle various complex tasks, yet issues such as hallucinations and weak object discrimination persist. Discriminative training, exemplified by models like CLIP, excels in zero-shot image-text classification and retrieval, yet struggles with complex scenarios requiring fine-grained semantic differentiation. This paper addresses these challenges by proposing a unified approach that integrates the strengths of both paradigms. Considering interleaved image-text sequences as the general format of input samples, we introduce a structure-induced training strategy that imposes semantic relationships between input …

Generative Semi-Supervised Graph Anomaly Detection, Hezhe Qiao, Qingsong Wen, Xiaoli Li, Ee-Peng Lim, Guansong Pang

Research Collection School Of Computing and Information Systems

This work considers a practical semi-supervised graph anomaly detection (GAD) scenario, where part of the nodes in a graph are known to be normal, contrasting to the extensively explored unsupervised setting with a fully unlabeled graph. We reveal that having access to the normal nodes, even just a small percentage of normal nodes, helps enhance the detection performance of existing unsupervised GAD methods when they are adapted to the semi-supervised setting. However, their utilization of these normal nodes is limited. In this paper we propose a novel Generative GAD approach (namely GGAD) for the semi-supervised scenario to better exploit the …

3d Snapshot: Invertible Embedding Of 3d Neural Representations In A Single Image, Yuqin Lu, Bailin Deng, Zhixuan Zhong, Tianle Zhang, Yuhui Quan, Hongmin Cai, Shengfeng He

Research Collection School Of Computing and Information Systems

3D neural rendering enables photo-realistic reconstruction of a specific scene by encoding discontinuous inputs into a neural representation. Despite the remarkable rendering results, the storage of network parameters is not transmission-friendly and not extendable to metaverse applications. In this paper, we propose an invertible neural rendering approach that enables generating an interactive 3D model from a single image (i.e., 3D Snapshot). Our idea is to distill a pre-trained neural rendering model (e.g., NeRF) into a visualizable image form that can then be easily inverted back to a neural network. To this end, we first present a neural image distillation method …

Mimicking To Dominate: Imitation Learning Strategies For Success In Multiagent Competitive Games, The Viet Bui, Tien Mai, Hong Thanh Nguyen

Research Collection School Of Computing and Information Systems

Training agents in multi-agent games presents significant challenges due to their intricate nature. These challenges are exacerbated by dynamics influenced not only by the environment but also by strategies of opponents. Existing methods often struggle with slow convergence and instability. To address these challenges, we harness the potential of imitation learning (IL) to comprehend and anticipate actions of the opponents, aiming to mitigate uncertainties with respect to the game dynamics. Our key contributions include: (i) a new multi-agent IL model for predicting next moves of the opponents --- our model works with hidden actions of opponents and local observations; (ii) …

Inverse Factorized Soft Q-Learning For Cooperative Multi-Agent Imitation Learning, The Viet Bui, Tien Mai, Thanh Nguyen

Research Collection School Of Computing and Information Systems

This paper concerns imitation learning (IL) in cooperative multi-agent systems.The learning problem under consideration poses several challenges, characterized by high-dimensional state and action spaces and intricate inter-agent dependencies. In a single-agent setting, IL was shown to be done efficiently via an inverse soft-Q learning process. However, extending this framework to a multi-agent context introduces the need to simultaneously learn both local value functions to capture local observations and individual actions, and a joint value function for exploiting centralized learning.In this work, we introduce a new multi-agent IL algorithm designed to address these challenges. Our approach enables thecentralized learning by leveraging …

Trustworthy Web3 Domains: A Framework For Digital Identity Verification, Yi Meng Lau, Ping Fan Ke

Research Collection School Of Computing and Information Systems

As decentralized applications evolve, digital identities represented through Web3 domain names gained prominence. This study addresses the challenges of establishing trust in Web3 domain names. The decentralized nature of Web3 introduces complexities in verifying domain name authenticity, making them targets for malicious activities such as cybersquatting and phishing. We propose a comprehensive framework that enhances traditional identification, authentication, and authorization processes by incorporating technological and social trust elements. This framework enables organizations and users to systematically assess the trustworthiness of Web3 domain names, offering a structured approach to managing digital identities in decentralized environments.

A Community-Driven Framework To Evaluate Factors Influencing Nft Buyers, Yi Meng Lau, Ping Fan Ke

Research Collection School Of Computing and Information Systems

This study examines the pivotal role of community-driven social dynamics in shaping non-fungible tokens (NFT) buyers’ purchasing intentions and the increasing social significance of digital assets. Building on prior research in online word-of-mouth, we propose a comprehensive framework that integrates technological, personal, social, economic, and political factors influencing NFT purchases. The framework emphasizes the importance of service providers, personalization, perceived value, and associated risks, while also providing insights into how online word-of-mouth and community dynamics impact these aspects. As a preliminary exploration, this research lays the foundation for future empirical studies to validate the framework and assess its applicability across …

Towards General Conceptual Model Editing Via Adversarial Representation Engineering, Yihao Zhang, Zeming Wei, Jun Sun, Meng Sun

Research Collection School Of Computing and Information Systems

Since the rapid development of Large Language Models (LLMs) has achieved remarkable success, understanding and rectifying their internal complex mechanisms has become an urgent issue. Recent research has attempted to interpret their behaviors through the lens of inner representation. However, developing practical and efficient methods for applying these representations for general and flexible model editing remains challenging. In this work, we explore how to leverage insights from representation engineering to guide the editing of LLMs by deploying a representation sensor as an editing oracle. We first identify the importance of a robust and reliable sensor during editing, then propose an …

Question-Attentive Review-Level Explanation For Neural Rating Regression, Trung Hoang Le, Hady Wirawan Lauw

Research Collection School Of Computing and Information Systems

Recommendation explanations help to improve their acceptance by end users. Explanations come in many different forms. One that is of interest here is presenting an existing review of the recommended item as the explanation. The challenge is in selecting a suitable review, which is customarily addressed by assessing the relative importance or “attention” of each review to the recommendation objective. Our focus is improving review-level explanation by leveraging additional information in the form of questions and answers (QA). The proposed framework employs QA in an attention mechanism that aligns reviews to various QAs of an item and assesses their contribution …

Harnessing The Power Of Ai-Instructor Collaborative Grading Approach: Topic-Based Effective Grading For Semi Open-Ended Multipart Questions, Phyo Yi Win Myint, Siaw Ling Lo, Yuhao Zhang

Research Collection School Of Computing and Information Systems

Semi open-ended multipart questions consist of multiple sub questions within a single question, requiring students to provide certain factual information while allowing them to express their opinion within a defined context. Human grading of such questions can be tedious, constrained by the marking scheme and susceptible to the subjective judgement of instructors. The emergence of large language models (LLMs) such as ChatGPT has significantly advanced the prospect of automatic grading in educational settings. This paper introduces a topic-based grading approach that harnesses LLM capabilities alongside a refined marking scheme to ensure fair and explainable assessment processes. The proposed approach involves …

Custom Permission Misconfigurations In Android: A Large-Scale Security Analysis, Rui Li, Wenrui Diao, Debin Gao

Research Collection School Of Computing and Information Systems

Android’s popularity is due to its openness and vast app ecosystem. Global developers can use Android Studio and rich Android APIs to create their apps. Within this ecosystem, Android permissions play a crucial role in managing access to resources, with system permissions controlled by system apps and custom permissions declared by third-party apps. However, the security of custom permissions has not received enough attention from the mobile security community, resulting in a lack of thorough evaluation of security practices for app developers using custom permissions. This study systematically evaluated the misconfiguration of custom permissions by Android app developers. It is …

Improving Environment Novelty Quantification For Effective Unsupervised Environment Design, Jayden Teoh, Wenjun Li, Pradeep Varakantham

Research Collection School Of Computing and Information Systems

Unsupervised Environment Design (UED) formalizes the problem of autocurricula through interactive training between a teacher agent and a student agent. The teacher generates new training environments with high learning potential, curating an adaptive curriculum that strengthens the student’s ability to handle unseen scenarios. Existing UED methods mainly rely on regret, a metric that measures the difference between the agent’s optimal and actual performance, to guide curriculum design. Regret-driven methods generate curricula that progressively increase environment complexity for the student but overlook environment novelty–a critical element for enhancing an agent’s generalizability. Measuring environment novelty is especially challenging due to the underspecified …

Delidar: Decoupling Lidars For Pervasive Spatial Computing, Kanatta Gamage Ramesh Darshana Rathnayake, Razat Sutradhar, Abbaas A. M. Nishar, Weerakoon Dulaj S., Ashwin Ashok, Archan Misra

Research Collection School Of Computing and Information Systems

Unbounded proliferation of LiDAR-equipped pervasive devices generates two challenges: (a) mutual interference among emitters and (b) significantly higher sensing energy overhead. We propose a fundamentally different approach for LiDAR sensing, in indoor spaces, that decouples the sensor’s emitter and receiver components. Our proposed approach, called DeLiDAR, centralizes the emitter functionality in one or more stationary nodes that continually emit pulses; this decoupling allows each mobile LiDAR sensor to be an ultra-low power, pure receiver unit consisting solely of passive multiple photodiodes. We explain how the emitter can utilize VLC-based encoding of its pulses to convey parameter settings that allow a …

Elevating Automated Software Maintenance Tasks With Large Language Models, Xin Zhou

Dissertations and Theses Collection (Open Access)

Software engineering involves many tasks across different phases such as requirements, design, implementation, testing, and maintenance. Among them, software maintenance is a crucial phase, typically accounting for more than half of the software life cycle's duration.
To boost developer productivity, in recent years, numerous research endeavors in software engineering have sought to automate certain software maintenance tasks through the application of machine learning techniques.
Since 2020, the emergence of advanced Large Language Models (LLMs) of code has opened new avenues for enhancing automated solutions in software maintenance.
This dissertation presents a series of works aimed at advancing automated solutions for …

Efficient Multiplicative-To-Additive Function From Joye-Libert Cryptosystem And Its Application To Threshold Ecdsa, Haiyang Xue, Ho Man Au, Mengling Liu, Yin Kwan Chan, Handong Cui, Xiang Xie, Hon Tsz Yuen, Chengru Zhang

Research Collection School Of Computing and Information Systems

Threshold ECDSA receives interest lately due to its widespread adoption in blockchain applications. A common building block of all leading constructions involves a secure conversion of multiplicative shares into additive ones, which is called the multiplicative-to-additive (MtA) function. MtA dominates the overall complexity of all existing threshold ECDSA constructions. Specifically, O(n2) invocations of MtA are required in the case of n active signers. Hence, improvement of MtA leads directly to significant improvements for all state-of-the-art threshold ECDSA schemes.In this paper, we design a novel MtA by revisiting the Joye-Libert (JL) cryptosystem. Specifically, we revisit JL encryption and propose a JL-based …

A Survey Of Ontology Expansion For Conversational Understanding, Jinggui Liang, Yuxia Wu, Yuan Fang, Hao Fei, Lizi Liao

Research Collection School Of Computing and Information Systems

In the rapidly evolving field of conversational AI, Ontology Expansion (OnExp) is crucial for enhancing the adaptability and robustness of conversational agents. Traditional models rely on static, predefined ontologies, limiting their ability to handle new and unforeseen user needs. This survey paper provides a comprehensive review of the state-of-the-art techniques in OnExp for conversational understanding. It categorizes the existing literature into three main areas: (1) New Intent Discovery, (2) New Slot-Value Discovery, and (3) Joint OnExp. By examining the methodologies, benchmarks, and challenges associated with these areas, we highlight several emerging frontiers in OnExp to improve agent performance in real-world …

Ultra-High Resolution Image Segmentation Via Locality-Aware Context Fusion And Alternating Local Enhancement, Wenxi Liu, Qi Li, Xindai Lin, Weixiang Yang, Shengfeng He, Yuanlong Yu

Research Collection School Of Computing and Information Systems

Ultra-high resolution image segmentation has raised increasing interests in recent years due to its realistic applications. In this paper, we innovate the widely used high-resolution image segmentation pipeline, in which an ultra-high resolution image is partitioned into regular patches for local segmentation and then the local results are merged into a high-resolution semantic mask. In particular, we introduce a novel locality-aware context fusion based segmentation model to process local patches, where the relevance between local patch and its various contexts are jointly and complementarily utilized to handle the semantic regions with large variations. Additionally, we present the alternating local enhancement …

Large Language Models For Software Engineering: A Systematic Literature Review, Xinyi Hou, Yanjie Zhao, Yue Liu, Zhou Yang, Kailong Wang, Li Li, Xiapu Luo, David Lo, John Grundy, Haoyu Wang

Research Collection School Of Computing and Information Systems

Large Language Models (LLMs) have significantly impacted numerous domains, including Software Engineering (SE). Many recent publications have explored LLMs applied to various SE tasks. Nevertheless, a comprehensive understanding of the application, effects, and possible limitations of LLMs on SE is still in its early stages. To bridge this gap, we conducted a Systematic Literature Review (SLR) on LLM4SE, with a particular focus on understanding how LLMs can be exploited to optimize processes and outcomes. We selected and analyzed 395 research articles from January 2017 to January 2024 to answer four key Research Questions (RQs). In RQ1, we categorize different LLMs …

Strength Lies In Differences! Improving Strategy Planning For Non-Collaborative Dialogues Via Diversified User Simulation, Tong Zhang, Chen Huang, Yang Deng, Hongru Liang, Jia Liu, Zujie Wen, Wenqiang Lei, Tat-Seng Chua

Research Collection School Of Computing and Information Systems

We investigate non-collaborative dialogue agents, which are expected to engage in strategic conversations with diverse users, for securing a mutual agreement that leans favorably towards the system’s objectives. This poses two main challenges for existing dialogue agents: 1) The inability to integrate user-specific characteristics into the strategic planning, and 2) The difficulty of training strategic planners that can be generalized to diverse users. To address these challenges, we propose TRIP to enhance the capability in tailored strategic planning, incorporating a user-aware strategic planning module and a population-based training paradigm. Through experiments on benchmark non-collaborative dialogue tasks, we demonstrate the effectiveness …

Thoughts To Target: Enhance Planning For Target-Driven Conversation, Zhonghua Zheng, Lizi Liao, Yang Deng, Ee-Peng Lim, Minlie Huang, Liqiang Nie

Research Collection School Of Computing and Information Systems

In conversational AI, large-scale models excel in various tasks but struggle with target-driven conversation planning. Current methods, such as chain-of-thought reasoning and tree-search policy learning techniques, either neglect plan rationality or require extensive human simulation procedures. Addressing this, we propose a novel two-stage framework, named EnPL, to improve the LLMs’ capability in planning conversations towards designated targets, including (1) distilling natural language plans from target-driven conversation corpus and (2) generating new plans with demonstration-guided in-context learning. Specifically, we first propose a filter approach to distill a high-quality plan dataset, ConvPlan1. With the aid of corresponding conversational data and support from …

Angels Or Demons: Investigating And Detecting Decentralized Financial Traps On Ethereum Smart Contracts, Jiachi Chen, Jiang Hu, Xin Xia, David Lo, John Grundy, Zhipeng Gao, Ting Chen

Research Collection School Of Computing and Information Systems

Decentralized Finance (DeFi) uses blockchain technologies to transform traditional financial activities into decentralized platforms that run without intermediaries and centralized institutions. Smart contracts are programs that run on the blockchain, and by utilizing smart contracts, developers can more easily develop DeFi applications. Some key features of smart contracts—self-executed and immutability—ensure the trustworthiness, transparency and efficiency of DeFi applications and have led to a fast-growing DeFi market. However, misbehaving developers can add traps or backdoor code snippets to a smart contract, which are hard for contract users to discover. We call these code snippets in a DeFi smart contract as “DeFi …

Ai And Creativity: Effects Of Culture And Task Emotiveness In Human-Ai Collaboration, Choon Ngee Tan

Dissertations and Theses Collection (Open Access)

Creativity is the driving force behind innovation, propelling individuals and societies toward progress by generating novel ideas and groundbreaking solutions. The emergence of generative AI models, exemplified by GPT-3, offers opportunities to enhance human creativity. This paper explores the potential for unprecedented breakthroughs through the synergy between human intuition and AI-driven creativity, providing practical guidance on leveraging AI to amplify creative capacities. Study 1 finds that AI models trained on data from the U.S. and Chinese cultures exhibit cultural norms, values and cognition of those cultures. Study 2 finds that when humans and AI models of the same culture collaborate …

Towards Trustworthy Recommendation Systems: Beyond Collaborative Filtering, Zhongzhou Liu, Zhongzhou

Dissertations and Theses Collection (Open Access)

Recommendation systems have been widely deployed in various scenarios and applications, such as e-commerce, social media, and streaming services. Recommendation systems have significantly influenced how we interact with various items in a wide range of platforms. They help users discover their preferred items and provide efficient and enjoyable experiences. They also help item providers and platforms to quickly find their potential customers, thus increasing the total revenue and user engagement.

The majority of existing recommendation systems merely focus on the matching between users and items, aiming for higher recommendation accuracy. Collaborative filtering is regarded as one of the most successful …

Contactless And Scalable Approaches For Human Health And Performance Sensing, Ngoc Doan Thu Tran

Dissertations and Theses Collection (Open Access)

Human health and performance sensing has been extensively studied, from physiology to mental health and movement analytics. However, typical approaches rely on invasive and contact sensors or require professional practitioners, limiting their scalability. For example, the gold standard for measuring heart rate is through an electrocardiogram (ECG), which requires multiple probes attached to the skin and is impractical for individuals with skin issues. Additionally, it typically needs to be performed in a hospital setting under the supervision of a trained cardiac physiologist. Depression detection often relies on the expertise of psychologists or psychiatrists. However, there is a shortage of these …

Food Computing: Domain Adaptation And Causal Inference, Qing Wang

Dissertations and Theses Collection (Open Access)

This dissertation addresses two challenges in food computing: food recognition and food image-to-recipe retrieval. The main research ideas are: (1) leveraging Large Language Models (LLMs) to augment food image representations to mitigate the combined challenges of domain gaps and data imbalance in fine-grained food recognition; (2) proposing a causal-theory inspired cross-modal representation learning formulation for reducing the bias caused by the emphasis on certain ingredients for cross-modal recipe retrieval; and (3) extending the framework to incorporate multiple confounding factors, particularly ingredients and cooking actions, allows for more comprehensive modeling of the food image-torecipe retrieval problem.

We first explore the challenges …

Law-Aware Autonomous Driving, Yang Sun

Dissertations and Theses Collection (Open Access)

Autonomous driving systems (ADSs) necessitate comprehensive testing prior to deployment in Autonomous Vehicles (AVs). High-fidelity simulators are crucial for this testing, as they can replicate a wide range of scenarios, including those that are difficult or dangerous to recreate in real-world conditions. While previous approaches have demonstrated that test cases can be generated automatically, they often focus on weak oracles (e.g., reaching the destination without collisions) and fail to assess whether the journey was conducted safely and in compliance with some complex property specifications such as traffic laws. In this dissertation, beyond assessing basic properties like energy consumption and proximity …

Uncovering Merchants’ Willingness To Wait In On-Demand Food Delivery Markets, Jian Liang, Ya Zhao, Hai Wang, Zuopeng Xiao, Jintao Ke

Research Collection School Of Computing and Information Systems

While traditional on-demand food delivery services help restaurants reach more customers and enable doorstep deliveries, they also come with drawbacks, such as high commission fees and limited control over the delivery process. White-label food delivery services have emerged as an alternative, ready-to-use platform for restaurants to arrange delivery for customer orders received through their applications or websites, without the constraints imposed by traditional on-demand food delivery platforms or the need to develop an in-house delivery operation. Although several studies have investigated consumer behavior when using traditional on-demand food delivery services, there is limited research on merchants’ behavior when adopting white-label …

Cirp: Cross‑Item Relational Pre‑Training For Multimodal Product Bundling, Yunshan Ma, Yingzhi He, Wenjun Zhong, Xiang Wang, Roger Zimmermann, Tat-Seng Chua

Research Collection School Of Computing and Information Systems

Product bundling has been a prevailing marketing strategy that is beneficial in the online shopping scenario. Effective product bundling methods depend on high-quality item representations capturing both the individual items' semantics and cross-item relations. However, previous item representation learning methods, either feature fusion or graph learning, suffer from inadequate cross-modal alignment and struggle to capture the cross-item relations for cold-start items. Multimodal pre-train models could be the potential solutions given their promising performance on various multimodal downstream tasks. However, the cross-item relations have been under-explored in the current multimodal pre-train models.To bridge this gap, we propose a novel and simple …

Don’T Just Say “I Don’T Know”! Self-Aligning Large Language Models For Responding To Unknown Questions With Explanations, Yang Deng, Yong Zhao, Moxin Li, See-Kiong Ng, Tat-Seng Chua

Research Collection School Of Computing and Information Systems

Despite the remarkable abilities of Large Language Models (LLMs) to answer questions, they often display a considerable level of overconfidence even when the question does not have a definitive answer. To avoid providing hallucinated answers to these unknown questions, existing studies typically investigate approaches to refusing to answer these questions. In this work, we propose a novel and scalable self-alignment method to utilize the LLM itself to enhance its response-ability to different types of unknown questions, being capable of not only refusing to answer but also providing explanation to the unanswerability of unknown questions. Specifically, the Self-Align method first employ …

Eyetraes : Fine-Grained, Low-Latency Eye Tracking Via Adaptive Event Slicing, Argha Sen, Panahetipola Mudiyanselage Nuwan Bandara, Ila Gokarn, Thivya Kandappu, Archan Misra

Research Collection School Of Computing and Information Systems

Eye-tracking technology has gained significant attention in recent years due to its wide range of applications in humancomputer interaction, virtual and augmented reality, and wearable health. Traditional RGB camera-based eye-tracking systems often struggle with poor temporal resolution and computational constraints, limiting their effectiveness in capturing rapid eye movements. To address these limitations, we propose EyeTrAES, a novel approach using neuromorphic event cameras for high-fidelity tracking of natural pupillary movement that shows significant kinematic variance. One of EyeTrAES’s highlights is the use of a novel adaptive windowing/slicing algorithm that ensures just the right amount of descriptive asynchronous event data accumulation within …