Artificial Intelligence and Robotics Commons^™

Modeling And Regulating A Ride-Sourcing Market Integrated With Vehicle Rental Services, Dong Mo, Hai Wang, Zeen Cai, W. Y. Szeto, Xiqun (Michael) Chen

Research Collection School Of Computing and Information Systems

With the popularity of on-demand ride services worldwide, ride-sourcing platforms must maintain an adequate fleet size and cope with growing travel demand. Recently, platforms have attempted to provide vehicle rental services to drivers who do not own cars, then recruited them to provide on demand ride services. This helps lower the entry barrier for drivers and offers another profitable business for platforms. From the government's perspective, however, it is challenging to coordinately regulate a ride-sourcing business and vehicle rental business. This paper proposes a bi-level optimization model to investigate how the government regulates the ride-sourcing market integrated with vehicle rental …

Reinforcement Learning Based Online Request Scheduling Framework For Workload-Adaptive Edge Deep Learning Inference, Xinrui Tan, Hongjia Li, Xiaofei Xie, Lu Guo, Nirwan Ansari, Xueqing Huang, Liming Wang, Zhen Xu, Yang Liu

Research Collection School Of Computing and Information Systems

The recent advances of deep learning in various mobile and Internet-of-Things applications, coupled with the emergence of edge computing, have led to a strong trend of performing deep learning inference on the edge servers located physically close to the end devices. This trend presents the challenge of how to meet the quality-of-service requirements of inference tasks at the resource-constrained network edge, especially under variable or even bursty inference workloads. Solutions to this challenge have not yet been reported in the related literature. In the present paper, we tackle this challenge by means of workload-adaptive inference request scheduling: in different workload …

Mvgamba : Unify 3d Content Generation As State Space Sequence Modeling, Xuanyu Yi, Zike Wu, Qiuhong Shen, Qingshan Xu, Pan Zhou, Joo-Hwee Lim, Shuicheng Yan, Xinchao Wang, Hanwang Zhang

Research Collection School Of Computing and Information Systems

Recent 3D large reconstruction models (LRMs) can generate high-quality 3D content in sub-seconds by integrating multi-view diffusion models with scalable multi-view reconstructors. Current works further leverage 3D Gaussian Splatting as 3D representation for improved visual quality and rendering efficiency. However, we observe that existing Gaussian reconstruction models often suffer from multi-view inconsistency and blurred textures. We attribute this to the compromise of multi-view information propagation in favor of adopting powerful yet computationally intensive architectures (e.g., Transformers). To address this issue, we introduce MVGamba, a general and lightweight Gaussian reconstruction model featuring a multi-view Gaussian reconstructor based on the RNN-like State …

Converting Vocal Performances Into Sheet Music Leveraging Large Language Models, Jinjing Jiang, Nicole Teo, Haibo Pen, Seng-Beng Ho, Zhaoxia Wang

Research Collection School Of Computing and Information Systems

Advanced natural language processing (NLP) models are increasingly applied in music composition and performance, particularly for generating vocal melodies and simulating singing voices. While NLP techniques have been effective in analyzing vocal performance data to assess quality and style, the automatic transcription of vocal performances into sheet music remains a significant challenge. Manual transcription tools often fall short due to the intricate dynamics of vocal expression. This study tackles the automation of vocal performance transcription into sheet music using innovative techniques, including large language models (LLMs). We propose a method to translate vocal audio input into display-ready sheet music effectively. …

Lova3 : Learning To Visual Question Answering, Asking And Assessment, Henry Hengyuan Zhao, Pan Zhou, Difei Gao, Bai Shou, Mike Zheng Shou

Research Collection School Of Computing and Information Systems

Question answering, asking, and assessment are three innate human traits crucial for understanding the world and acquiring knowledge. By enhancing these capabilities, humans can more effectively utilize data, leading to better comprehension and learning outcomes. Current Multimodal Large Language Models (MLLMs) primarily focus on question answering, often neglecting the full potential of questioning and assessment skills. Inspired by the human learning mechanism, we introduce LOVA3 , an innovative framework named “Learning tO Visual question Answering, Asking and Assessment,” designed to equip MLLMs with these additional capabilities. Our approach involves the creation of two supplementary training tasks GenQA and EvalQA, aiming …

Unified Generative And Discriminative Training For Multi-Modal Large Language Models, Wei Chow, Juncheng Li, Kaihang Pan, Qifan Yu, Hao Fei, Zhiqi Ge, Shuai Yang, Siliang Teng, Hanwang Zhang, Qianru Sun

Research Collection School Of Computing and Information Systems

In recent times, Vision-Language Models (VLMs) have been trained under two predominant paradigms. Generative training has enabled Multimodal Large Language Models (MLLMs) to tackle various complex tasks, yet issues such as hallucinations and weak object discrimination persist. Discriminative training, exemplified by models like CLIP, excels in zero-shot image-text classification and retrieval, yet struggles with complex scenarios requiring fine-grained semantic differentiation. This paper addresses these challenges by proposing a unified approach that integrates the strengths of both paradigms. Considering interleaved image-text sequences as the general format of input samples, we introduce a structure-induced training strategy that imposes semantic relationships between input …

Automating Maritime Risk Data Collection And Identification Leveraging Large Language Models, Donghao Huang, Xiuju Fu, Xiaofeng Yin, Haibo Pen, Zhaoxia Wang

Research Collection School Of Computing and Information Systems

Maritime risk research is crucial yet challenging for improving safety, efficiency, and sustainability in maritime operations. This paper presents an innovative method for automating the collection and identification of risk data related to global maritime risks from news sources, addressing the limitations of traditional manual methods. To evaluate the proposed method, different learning-based models, including conventional machine learning approaches and advanced Large Language Models (LLMs) such as GPT-4 and LLaMA-3.1, are comprehensively studied for comparison. In addition, not only do we use popular evaluation metrics to assess the proposed method, but we also introduce a new evaluation metric, called the …

Ai And Creativity: Effects Of Culture And Task Emotiveness In Human-Ai Collaboration, Choon Ngee Tan

Dissertations and Theses Collection (Open Access)

Creativity is the driving force behind innovation, propelling individuals and societies toward progress by generating novel ideas and groundbreaking solutions. The emergence of generative AI models, exemplified by GPT-3, offers opportunities to enhance human creativity. This paper explores the potential for unprecedented breakthroughs through the synergy between human intuition and AI-driven creativity, providing practical guidance on leveraging AI to amplify creative capacities. Study 1 finds that AI models trained on data from the U.S. and Chinese cultures exhibit cultural norms, values and cognition of those cultures. Study 2 finds that when humans and AI models of the same culture collaborate …

Food Computing: Domain Adaptation And Causal Inference, Qing Wang

Dissertations and Theses Collection (Open Access)

This dissertation addresses two challenges in food computing: food recognition and food image-to-recipe retrieval. The main research ideas are: (1) leveraging Large Language Models (LLMs) to augment food image representations to mitigate the combined challenges of domain gaps and data imbalance in fine-grained food recognition; (2) proposing a causal-theory inspired cross-modal representation learning formulation for reducing the bias caused by the emphasis on certain ingredients for cross-modal recipe retrieval; and (3) extending the framework to incorporate multiple confounding factors, particularly ingredients and cooking actions, allows for more comprehensive modeling of the food image-torecipe retrieval problem.

We first explore the challenges …

The Epistemic Role Of Ai Decision Support Systems: Neither Superiors, Nor Inferiors, Nor Peers, Rand Hirmiz

Research Collection School of Social Sciences

Despite the importance of discussions over the epistemic role that artificially intelligent decision support systems ought to play, there is currently a lack of these discussions in both the AI literature and the epistemology literature. My goal in this paper is to rectify this by proposing an account of the epistemic role of AI decision support systems in medicine and discussing what this epistemic role means with regard to how these systems ought to be utilized. In particular, I argue that AI decision support systems are not epistemic superiors, inferiors, or peers. Instead, I recommend that they be classified in …

Against The Substitutive Approach To Ai In Healthcare, Rand Hirmiz

Research Collection School of Social Sciences

Paul Bloom has famously argued against the need for empathy in clinicians, while Sally Dalton-Brown has argued that AI need not be capable of empathy to be a good carer. In this paper, the capacity for AI to substitute for human clinicians is assessed from a bioethical perspective, primarily through the evaluation of the arguments put forth by Dalton-Brown and Bloom concerning empathy in healthcare. In opposition to both Bloom and Dalton-Brown, this paper argues that (1) empathy is essential to providing good care or deep care (that is, care that goes beyond the mere fulfilment of medical tasks), (2) …

Class Name Guided Out-Of-Scope Intent Classification, Chandan Gautam, Sethupathy Parameswaran, Aditya Kane, Yuan Fang, Savitha Ramasamy, Suresh Sundaram, Sunil Kumar Sahu, Xiaoli Li

Research Collection School Of Computing and Information Systems

The paper introduces Semantics of Class Labelbased Unsupervised Out of Scope Intent Detection (SCOOS), a novel method aimed at enhancing out-of-scope (OOS) intent classification in task-oriented dialogue systems. Unlike prior approaches that rely solely on indomain (ID) data features, SCOOS leverages semantic cues embedded in class labels to improve classification accuracy. The method entails forming a compact feature space centered around the semantics of class labels by minimizing losses between ID features and class names. SCOOS achieves this by creating a compact feature space centered around class label semantics, achieved through minimizing losses between in-domain (ID) features and class names. …

Ultra-High Resolution Image Segmentation Via Locality-Aware Context Fusion And Alternating Local Enhancement, Wenxi Liu, Qi Li, Xindai Lin, Weixiang Yang, Shengfeng He, Yuanlong Yu

Research Collection School Of Computing and Information Systems

Ultra-high resolution image segmentation has raised increasing interests in recent years due to its realistic applications. In this paper, we innovate the widely used high-resolution image segmentation pipeline, in which an ultra-high resolution image is partitioned into regular patches for local segmentation and then the local results are merged into a high-resolution semantic mask. In particular, we introduce a novel locality-aware context fusion based segmentation model to process local patches, where the relevance between local patch and its various contexts are jointly and complementarily utilized to handle the semantic regions with large variations. Additionally, we present the alternating local enhancement …

Revisiting Conversation Discourse For Dialogue Disentanglement, Bobo Li, Hao Fei, Fei Li, Shengqiong Wu, Lizi Liao, Yinwei Wei, Tat-Seng Chua, Donghong Ji

Research Collection School Of Computing and Information Systems

Dialogue disentanglement aims to detach the chronologically ordered utterances into several independent sessions. Conversation utterances are essentially organized and described by the underlying discourse, and thus dialogue disentanglement requires the full understanding and harnessing of the intrinsic discourse attribute. In this article, we propose enhancing dialogue disentanglement by taking full advantage of the dialogue discourse characteristics. First of all, in feature encoding stage, we construct the heterogeneous graph representations to model the various dialogue-specific discourse structural features, including the static speaker-role structures (i.e., speaker-utterance and speaker-mentioning structure) and the dynamic contextual structures (i.e., the utterance-distance and partial-replying structure). We then …

Large Language Models For Software Engineering: A Systematic Literature Review, Xinyi Hou, Yanjie Zhao, Yue Liu, Zhou Yang, Kailong Wang, Li Li, Xiapu Luo, David Lo, John Grundy, Haoyu Wang

Research Collection School Of Computing and Information Systems

Large Language Models (LLMs) have significantly impacted numerous domains, including Software Engineering (SE). Many recent publications have explored LLMs applied to various SE tasks. Nevertheless, a comprehensive understanding of the application, effects, and possible limitations of LLMs on SE is still in its early stages. To bridge this gap, we conducted a Systematic Literature Review (SLR) on LLM4SE, with a particular focus on understanding how LLMs can be exploited to optimize processes and outcomes. We selected and analyzed 395 research articles from January 2017 to January 2024 to answer four key Research Questions (RQs). In RQ1, we categorize different LLMs …

Cirp: Cross‑Item Relational Pre‑Training For Multimodal Product Bundling, Yunshan Ma, Yingzhi He, Wenjun Zhong, Xiang Wang, Roger Zimmermann, Tat-Seng Chua

Research Collection School Of Computing and Information Systems

Product bundling has been a prevailing marketing strategy that is beneficial in the online shopping scenario. Effective product bundling methods depend on high-quality item representations capturing both the individual items' semantics and cross-item relations. However, previous item representation learning methods, either feature fusion or graph learning, suffer from inadequate cross-modal alignment and struggle to capture the cross-item relations for cold-start items. Multimodal pre-train models could be the potential solutions given their promising performance on various multimodal downstream tasks. However, the cross-item relations have been under-explored in the current multimodal pre-train models.To bridge this gap, we propose a novel and simple …

Mm‑Forecast: A Multimodal Approach To Temporal Event Forecasting With Large Language Models, Haoxuan Li, Zhengmao Yang, Yunshan Ma, Yi Bin, Yang Yang, Tat-Seng Chua

Research Collection School Of Computing and Information Systems

We study an emerging and intriguing problem of multimodal temporal event forecasting with large language models. Compared to using text or graph modalities, the investigation of utilizing images for temporal event forecasting has not been fully explored, especially in the era of large language models (LLMs). To bridge this gap, we are particularly interested in two key questions of: 1) why images will help in temporal event forecasting, and 2) how to integrate images into the LLM-based forecasting framework. To answer these research questions, we propose to identify two essential functions that images play in the scenario of temporal event …

Cas: Fusing Dnn Optimization & Adaptive Sensing For Energy-Efficient Multi-Modal Inference, Dulanga Weerakoon, Vigneshwaran Subbaraju, Joo Hwee Lim, Archan Misra

Research Collection School Of Computing and Information Systems

Intelligent virtual agents are used to accomplish complex multi-modal tasks such as human instruction comprehension in mixed-reality environments by increasingly adopting richer, energy-intensive sensors and processing pipelines. In such applications, the context for activating sensors and processing blocks required to accomplish a given task instance is usually manifested via multiple sensing modes. Based on this observation, we introduce a novel Commit-and-Switch ( CAS ) paradigm that simultaneously seeks to reduce both sensing and processing energy. In CAS , we first commit to a low-energy computational pipeline with a subset of available sensors. Then, the task context estimated by this pipeline …

Strength Lies In Differences! Improving Strategy Planning For Non-Collaborative Dialogues Via Diversified User Simulation, Tong Zhang, Chen Huang, Yang Deng, Hongru Liang, Jia Liu, Zujie Wen, Wenqiang Lei, Tat-Seng Chua

Research Collection School Of Computing and Information Systems

We investigate non-collaborative dialogue agents, which are expected to engage in strategic conversations with diverse users, for securing a mutual agreement that leans favorably towards the system’s objectives. This poses two main challenges for existing dialogue agents: 1) The inability to integrate user-specific characteristics into the strategic planning, and 2) The difficulty of training strategic planners that can be generalized to diverse users. To address these challenges, we propose TRIP to enhance the capability in tailored strategic planning, incorporating a user-aware strategic planning module and a population-based training paradigm. Through experiments on benchmark non-collaborative dialogue tasks, we demonstrate the effectiveness …

Ask-Before-Plan : Proactive Language Agents For Real-World Planning, Xuan Zhang, Yang Deng, Zifeng Ren, See-Kiong Ng, Tat-Seng Chua

Research Collection School Of Computing and Information Systems

The evolution of large language models (LLMs) has enhanced the planning capabilities of language agents in diverse real-world scenarios. Despite these advancements, the potential of LLM-powered agents to comprehend ambiguous user instructions for reasoning and decision-making is still under exploration. In this work, we introduce a new task, Proactive Agent Planning, which requires language agents to predict clarification needs based on user-agent conversation and agent-environment interaction, invoke external tools to collect valid information, and generate a plan to fulfill the user's demands. To study this practical problem, we establish a new benchmark dataset, Ask-before-Plan. To tackle the deficiency of LLMs …

Thoughts To Target: Enhance Planning For Target-Driven Conversation, Zhonghua Zheng, Lizi Liao, Yang Deng, Ee-Peng Lim, Minlie Huang, Liqiang Nie

Research Collection School Of Computing and Information Systems

In conversational AI, large-scale models excel in various tasks but struggle with target-driven conversation planning. Current methods, such as chain-of-thought reasoning and tree-search policy learning techniques, either neglect plan rationality or require extensive human simulation procedures. Addressing this, we propose a novel two-stage framework, named EnPL, to improve the LLMs’ capability in planning conversations towards designated targets, including (1) distilling natural language plans from target-driven conversation corpus and (2) generating new plans with demonstration-guided in-context learning. Specifically, we first propose a filter approach to distill a high-quality plan dataset, ConvPlan1. With the aid of corresponding conversational data and support from …

National Use Of Artificial Intelligence For Eye Screening In Singapore, Dinesh Visva Gunasekeran, Steven Miller, Wynne Hsu, Mong Li, Tym Hon Wong, Mun Tuck Lee, Ecosse Lamoureau, Daniel Shu Wei Ting, Gavin Siew Wei Tan, Tien-Yin Wong

Research Collection School Of Computing and Information Systems

Diabetes is a major health care challenge, affecting 10% of the global population. One third of patients with diabetes have an ocular complication known as diabetic retinopathy (DR). DR progression to manifestations such as vision-threatening diabetic retinopathy (VTDR) remains the leading cause of blindness in working-aged adults. Yearly DR screening is a universally recommended practice in primary care settings for patients with diabetes, but it is often difficult to implement due to a lack of staffing and screening capacity in primary care. This case study highlights our experience with developing a medical artificial intelligence (AI) software-as-a-medical-device (SaMD) solution for DR …

Don’T Just Say “I Don’T Know”! Self-Aligning Large Language Models For Responding To Unknown Questions With Explanations, Yang Deng, Yong Zhao, Moxin Li, See-Kiong Ng, Tat-Seng Chua

Research Collection School Of Computing and Information Systems

Despite the remarkable abilities of Large Language Models (LLMs) to answer questions, they often display a considerable level of overconfidence even when the question does not have a definitive answer. To avoid providing hallucinated answers to these unknown questions, existing studies typically investigate approaches to refusing to answer these questions. In this work, we propose a novel and scalable self-alignment method to utilize the LLM itself to enhance its response-ability to different types of unknown questions, being capable of not only refusing to answer but also providing explanation to the unanswerability of unknown questions. Specifically, the Self-Align method first employ …

Selective Annotation Via Data Allocation: These Data Should Be Triaged To Experts For Annotation Rather Than The Model, Chen Huang, Yang Deng, Wenqiang Lei, Jiancheng Lv, Ido Dagan

Research Collection School Of Computing and Information Systems

To obtain high-quality annotations under limited budget, semi-automatic annotation methods are commonly used, where a portion of the data is annotated by experts and a model is then trained to complete the annotations for the remaining data. However, these methods mainly focus on selecting informative data for expert annotations to improve the model predictive ability (i.e., triage-to-human data), while the rest of the data is indiscriminately assigned to model annotation (i.e., triage-to-model data). This may lead to inefficiencies in budget allocation for annotations, as easy data that the model could accurately annotate may be unnecessarily assigned to the expert, and …

Beyond Persuasion : Towards Conversational Recommender System With Credible Explanations, Peixin Qin, Chen Huang, Yang Deng, Wenqiang Lei, Tat-Seng Chua

Research Collection School Of Computing and Information Systems

With the aid of large language models, current conversational recommender system (CRS) has gaining strong abilities to persuade users to accept recommended items. While these CRSs are highly persuasive, they can mislead users by incorporating incredible information in their explanations, ultimately damaging the long-term trust between users and the CRS. To address this, we propose a simple yet effective method, called PC-CRS, to enhance the credibility of CRS’s explanations during persuasion. It guides the explanation generation through our proposed credibility-aware persuasive strategies and then gradually refines explanations via post-hoc self-reflection. Experimental results demonstrate the efficacy of PC-CRS in promoting persuasive …

Experience As Source For Anticipation And Planning : Experiential Policy Learning For Target-Driven Recommendation Dialogues, Quang Huy Dao, Yang Deng, Khanh-Huyen Bui, Dung D. Le, Lizi Liao

Research Collection School Of Computing and Information Systems

Target-driven recommendation dialogues present unique challenges in dialogue management due to the necessity of anticipating user interactions for successful conversations. Current methods face significant limitations: (I) inadequate capabilities for conversation anticipation, (II) computational inefficiencies due to costly simulations, and (III) neglect of valuable past dialogue experiences. To address these limitations, we propose a new framework, Experiential Policy Learning (EPL), for enhancing such dialogues. EPL embodies the principle of Learning From Experience, facilitating anticipation with an experiential scoring function that estimates dialogue state potential using similar past interactions stored in long-term memory. To demonstrate its flexibility, we introduce Tree-structured EPL (T-EPL) …

A Survey Of Ontology Expansion For Conversational Understanding, Jinggui Liang, Yuxia Wu, Yuan Fang, Hao Fei, Lizi Liao

Research Collection School Of Computing and Information Systems

In the rapidly evolving field of conversational AI, Ontology Expansion (OnExp) is crucial for enhancing the adaptability and robustness of conversational agents. Traditional models rely on static, predefined ontologies, limiting their ability to handle new and unforeseen user needs. This survey paper provides a comprehensive review of the state-of-the-art techniques in OnExp for conversational understanding. It categorizes the existing literature into three main areas: (1) New Intent Discovery, (2) New Slot-Value Discovery, and (3) Joint OnExp. By examining the methodologies, benchmarks, and challenges associated with these areas, we highlight several emerging frontiers in OnExp to improve agent performance in real-world …

Consecutive Batch Model Editing With Hook Layers, Shuaiyi Li, Yang Deng, Deng Cai, Hongyuan Lu, Liang Chen, Wai Lam

Research Collection School Of Computing and Information Systems

As the typical retraining paradigm is unacceptably time- and resource-consuming, researchers are turning to model editing to find an effective way that supports both consecutive and batch scenarios to edit the model behavior directly. Despite all these practical expectations, existing model editing methods fail to realize all of them. Furthermore, the memory demands for such sequential model editing approaches tend to be prohibitive, frequently necessitating an external memory that grows incrementally over time. To cope with these challenges, we propose CoachHooK, a model editing method that simultaneously supports sequential and batch editing. CoachHooK is memory-friendly as it only needs a …

Context-Aware Adapter Tuning For Few-Shot Relation Learning In Knowledge Graphs, Ran Liu, Zhongzhou Liu, Xiaoli Li, Yuan Fang

Research Collection School Of Computing and Information Systems

Knowledge graphs (KGs) are instrumental in various real-world applications, yet they often suffer from incompleteness due to missing relations. To predict instances for novel relations with limited training examples, few-shot relation learning approaches have emerged, utilizing techniques such as meta-learning. However, the assumption is that novel relations in meta-testing and base relations in meta-training are independently and identically distributed, which may not hold in practice. To address the limitation, we propose RelAdapter, a context-aware adapter for few-shot relation learning in KGs designed to enhance the adaptation process in meta-learning. First, RelAdapter is equipped with a lightweight adapter module that facilitates …

Dc-Instruct : An Effective Framework For Generative Multi-Intent Spoken Language Understanding, Bowen Xing, Lizi Liao, Minlie Huang

Research Collection School Of Computing and Information Systems

In the realm of multi-intent spoken language understanding, recent advancements have leveraged the potential of prompt learning frameworks. However, critical gaps exist in these frameworks: the lack of explicit modeling of dual-task dependencies and the oversight of task-specific semantic differences among utterances. To address these shortcomings, we propose DC-Instruct, a novel generative framework based on Dual-task Inter-dependent Instructions (DII) and Supervised Contrastive Instructions (SCI). Specifically, DII guides large language models (LLMs) to generate labels for one task based on the other task’s labels, thereby explicitly capturing dual-task inter-dependencies. Moreover, SCI leverages utterance semantics differences by guiding LLMs to determine whether …