Open Access. Powered by Scholars. Published by Universities.®

Artificial Intelligence and Robotics Commons™

Open Access. Powered by Scholars. Published by Universities.®

Research Collection School Of Computing and Information Systems

Discipline
Keyword
Publication Year
File Type

Articles 931 - 960 of 1664

Full-Text Articles in Artificial Intelligence and Robotics

Heterogeneous Attentions For Solving Pickup And Delivery Problem Via Deep Reinforcement Learning, Jingwen Li, Liang Xin, Zhiguang Cao, Andrew Lim, Wen Song, Jie Zhang Mar 2022

Heterogeneous Attentions For Solving Pickup And Delivery Problem Via Deep Reinforcement Learning, Jingwen Li, Liang Xin, Zhiguang Cao, Andrew Lim, Wen Song, Jie Zhang

Research Collection School Of Computing and Information Systems

Recently, there is an emerging trend to apply deep reinforcement learning to solve the vehicle routing problem (VRP), where a learnt policy governs the selection of next node for visiting. However, existing methods could not handle well the pairing and precedence relationships in the pickup and delivery problem (PDP), which is a representative variant of VRP. To address this challenging issue, we leverage a novel neural network integrated with a heterogeneous attention mechanism to empower the policy in deep reinforcement learning to automatically select the nodes. In particular, the heterogeneous attention mechanism specifically prescribes attentions for each role of the …

Go to article

The Ai Chip Race, Guansong Pang Mar 2022

The Ai Chip Race, Guansong Pang

Research Collection School Of Computing and Information Systems

The strong demand of computing power for artificial intelligence (AI) and machine learning is accelerating the race to develop cheaper and faster AI chips. The AI chip market was valued 10.6 billion in 2021 and the total revenue is expected to reach 79.8 billion by 2027. To be part of the market, tech giants from different countries have been successively joining the race, while AI chip startups attracting billions of dollars are taking off like a rocket.

Go to article

Innovative Human Motion Sensing With Earbuds, Dong Ma, Andrea Ferlini, Cecilia Mascolo Mar 2022

Innovative Human Motion Sensing With Earbuds, Dong Ma, Andrea Ferlini, Cecilia Mascolo

Research Collection School Of Computing and Information Systems

Earbuds, ear-worn wearables, have attracted growing attention from both industry and academia. This trend has witnessed manufacturers embedding multiple sensors on earbuds to enrich their functionalities. For example, Apple AirPods, Sony WF-1000XM3, and Bose QuietControl 30, have been equipped with accelerometers for tapping interaction or multiple microphones for noise cancellation. On the other hand, the research community regards earbuds as a powerful personal-scale human sensing and computing platform. By integrating sensors like PPG, barometer, and ultrasonic sensors, researchers have been devising a plethora of earable sensing applications, such as blood pressure monitoring [1], facial expression recognition [2], and authentication [3].

Go to article

Towards Efficient Annotations For A Human-Ai Collaborative, Clinical Decision Support System: A Case Study On Physical Stroke Rehabilitation Assessment, Min Hun Lee, Daniel P. Siewiorek, Asim Smailagic, Alexandre Bernardino, Sergi Bermúdez I Badia Mar 2022

Towards Efficient Annotations For A Human-Ai Collaborative, Clinical Decision Support System: A Case Study On Physical Stroke Rehabilitation Assessment, Min Hun Lee, Daniel P. Siewiorek, Asim Smailagic, Alexandre Bernardino, Sergi Bermúdez I Badia

Research Collection School Of Computing and Information Systems

Artificial intelligence (AI) and machine learning (ML) algorithms are increasingly being explored to support various decision-making tasks in health (e.g. rehabilitation assessment). However, the development of such AI/ML-based decision support systems is challenging due to the expensive process to collect an annotated dataset. In this paper, we describe the development process of a human-AI collaborative, clinical decision support system that augments an ML model with a rule-based (RB) model from domain experts. We conducted its empirical evaluation in the context of assessing physical stroke rehabilitation with the dataset of three exercises from 15 post-stroke survivors and therapists. Our results bring …

Go to article

Debiasing Nlu Models Via Causal Intervention And Counterfactual Reasoning, Bing Tian, Yixin Cao, Yong Zhang, Chunxiao Xing Mar 2022

Debiasing Nlu Models Via Causal Intervention And Counterfactual Reasoning, Bing Tian, Yixin Cao, Yong Zhang, Chunxiao Xing

Research Collection School Of Computing and Information Systems

Recent studies have shown that strong Natural Language Understanding (NLU) models are prone to relying on annotation biases of the datasets as a shortcut, which goes against the underlying mechanisms of the task of interest. To reduce such biases, several recent works introduce debiasing methods to regularize the training process of targeted NLU models. In this paper, we provide a new perspective with causal inference to fnd out the bias. On the one hand, we show that there is an unobserved confounder for the natural language utterances and their respective classes, leading to spurious correlations from training data. To remove …

Go to article

Deconfounded Visual Grounding, Jianqiang Huang, Yu Qin, Jiaxin Qi, Qianru Sun, Hanwang Zhang Mar 2022

Deconfounded Visual Grounding, Jianqiang Huang, Yu Qin, Jiaxin Qi, Qianru Sun, Hanwang Zhang

Research Collection School Of Computing and Information Systems

We focus on the confounding bias between language and location in the visual grounding pipeline, where we find that the bias is the major visual reasoning bottleneck. For example, the grounding process is usually a trivial languagelocation association without visual reasoning, e.g., grounding any language query containing sheep to the nearly central regions, due to that most queries about sheep have groundtruth locations at the image center. First, we frame the visual grounding pipeline into a causal graph, which shows the causalities among image, query, target location and underlying confounder. Through the causal graph, we know how to break the …

Go to article

Neuron Coverage-Guided Domain Generalization, Chris Xing Tian, Haoliang Li, Xiaofei Xie, Yang Liu, Shiqi Wang Mar 2022

Neuron Coverage-Guided Domain Generalization, Chris Xing Tian, Haoliang Li, Xiaofei Xie, Yang Liu, Shiqi Wang

Research Collection School Of Computing and Information Systems

This paper focuses on the domain generalization task where domain knowledge is unavailable, and even worse, only samples from a single domain can be utilized during training. Our motivation originates from the recent progresses in deep neural network (DNN) testing, which has shown that maximizing neuron coverage of DNN can help to explore possible defects of DNN (i.e.,misclassification). More specifically, by treating the DNN as a program and each neuron as a functional point of the code, during the network training we aim to improve the generalization capability by maximizing the neuron coverage of DNN with the gradient similarity regularization …

Go to article

State Graph Reasoning For Multimodal Conversational Recommendation, Yuxia Wu, Lizi Liao, Gangyi Zhang, Wenqiang Lei, Guoshuai Zhao, Xueming Qian, Tat-Seng Chua Mar 2022

State Graph Reasoning For Multimodal Conversational Recommendation, Yuxia Wu, Lizi Liao, Gangyi Zhang, Wenqiang Lei, Guoshuai Zhao, Xueming Qian, Tat-Seng Chua

Research Collection School Of Computing and Information Systems

Conversational recommendation system (CRS) attracts increasing attention in various application domains such as retail and travel. It offers an effective way to capture users’ dynamic preferences with multi-turn conversations. However, most current studies center on the recommendation aspect while over-simplifying the conversation process. The negligence of complexity in data structure and conversation flow hinders their practicality and utility. In reality, there exist various relationships among slots and values, while users’ requirements may dynamically adjust or change. Moreover, the conversation often involves visual modality to facilitate the conversation. These actually call for a more advanced internal state representation of the dialogue …

Go to article

On The Influence Of Biases In Bug Localization: Evaluation And Benchmark, Ratnadira Widyasari, Stefanus Agus Haryono, Ferdian Thung, Jieke Shi, Constance Tan, Fiona Wee, Jack Phan, David Lo Mar 2022

On The Influence Of Biases In Bug Localization: Evaluation And Benchmark, Ratnadira Widyasari, Stefanus Agus Haryono, Ferdian Thung, Jieke Shi, Constance Tan, Fiona Wee, Jack Phan, David Lo

Research Collection School Of Computing and Information Systems

Bug localization is the task of identifying parts of thesource code that needs to be changed to resolve a bug report.As this task is difficult, automatic bug localization tools havebeen proposed. The development and evaluation of these toolsrely on the availability of high-quality bug report datasets. In2014, Kochhar et al. identified three biases in datasets used toevaluate bug localization techniques: (1) misclassified bug report,(2) already localized bug report, and (3) incorrect ground truthfile in a bug report. They reported that already localized bugreports statistically significantly and substantially impact buglocalization results, and thus should be removed. However, theirevaluation is still limited, …

Go to article

Sample-Efficient Iterative Lower Bound Optimization Of Deep Reactive Policies For Planning In Continuous Mdps, Siow Meng Low, Akshat Kumar, Scott Sanner Mar 2022

Sample-Efficient Iterative Lower Bound Optimization Of Deep Reactive Policies For Planning In Continuous Mdps, Siow Meng Low, Akshat Kumar, Scott Sanner

Research Collection School Of Computing and Information Systems

Recent advances in deep learning have enabled optimization of deep reactive policies (DRPs) for continuous MDP planning by encoding a parametric policy as a deep neural network and exploiting automatic differentiation in an end-toend model-based gradient descent framework. This approach has proven effective for optimizing DRPs in nonlinear continuous MDPs, but it requires a large number of sampled trajectories to learn effectively and can suffer from high variance in solution quality. In this work, we revisit the overall model-based DRP objective and instead take a minorizationmaximization perspective to iteratively optimize the DRP w.r.t. a locally tight lower-bounded objective. This novel …

Go to article

Interpretable Knowledge Tracing: Simple And Efficient Student Modeling With Causal Relations, Sein Minn, Jill-Jênn Vie, Koh Takeuchi, Feida Zhu Mar 2022

Interpretable Knowledge Tracing: Simple And Efficient Student Modeling With Causal Relations, Sein Minn, Jill-Jênn Vie, Koh Takeuchi, Feida Zhu

Research Collection School Of Computing and Information Systems

Intelligent Tutoring Systems have become critically important in future learning environments. Knowledge Tracing (KT) is a crucial part of that system. It is about inferring the skill mastery of students and predicting their performance to adjust the curriculum accordingly. Deep Learning based models like Deep Knowledge Tracing (DKT) and Dynamic Key-Value Memory Network (DKVMN) have shown significant predictive performance compared with traditional models like Bayesian Knowledge Tracing (BKT) and Performance Factors Analysis (PFA). However, it is difficult to extract psychologically meaningful explanations from the tens of thousands of parameters in neural networks, that would relate to cognitive theory. There are …

Go to article

Riconv++: Effective Rotation Invariant Convolutions For 3d Point Clouds Deep Learning, Zhiyuan Zhang, Binh-Son Hua, Sai-Kit Yeung Mar 2022

Riconv++: Effective Rotation Invariant Convolutions For 3d Point Clouds Deep Learning, Zhiyuan Zhang, Binh-Son Hua, Sai-Kit Yeung

Research Collection School Of Computing and Information Systems

3D point clouds deep learning is a promising field of research that allows a neural network to learn features of point clouds directly, making it a robust tool for solving 3D scene understanding tasks. While recent works show that point cloud convolutions can be invariant to translation and point permutation, investigations of the rotation invariance property for point cloud convolution has been so far scarce. Some existing methods perform point cloud convolutions with rotation-invariant features, existing methods generally do not perform as well as translation-invariant only counterpart. In this work, we argue that a key reason is that compared to …

Go to article

Deep Learning For Anomaly Detection: A Review, Guansong Pang, Chunhua Shen, Longbing Cao, Anton Van Den Hengel Mar 2022

Deep Learning For Anomaly Detection: A Review, Guansong Pang, Chunhua Shen, Longbing Cao, Anton Van Den Hengel

Research Collection School Of Computing and Information Systems

Anomaly detection, a.k.a. outlier detection or novelty detection, has been a lasting yet active research area in various research communities for several decades. There are still some unique problem complexities and challenges that require advanced approaches. In recent years, deep learning enabled anomaly detection, i.e., deep anomaly detection, has emerged as a critical direction. This article surveys the research of deep anomaly detection with a comprehensive taxonomy, covering advancements in 3 high-level categories and 11 fine-grained categories of the methods. We review their key intuitions, objective functions, underlying assumptions, advantages, and disadvantages and discuss how they address the aforementioned challenges. …

Go to article

Viral Pneumonia Screening On Chest X-Rays Using Confidence-Aware Anomaly Detection, Jianpeng Zhang, Yutong Xie, Guansong Pang, Zhibin Liao, Johan Verjans, Wenxing Li, Zongji Sun, Jian He, Yi Li, Chunhua Shen, Yong Xia Mar 2022

Viral Pneumonia Screening On Chest X-Rays Using Confidence-Aware Anomaly Detection, Jianpeng Zhang, Yutong Xie, Guansong Pang, Zhibin Liao, Johan Verjans, Wenxing Li, Zongji Sun, Jian He, Yi Li, Chunhua Shen, Yong Xia

Research Collection School Of Computing and Information Systems

Clusters of viral pneumonia occurrences over a short period may be a harbinger of an outbreak or pandemic. Rapid and accurate detection of viral pneumonia using chest X-rays can be of significant value for large-scale screening and epidemic prevention, particularly when other more sophisticated imaging modalities are not readily accessible. However, the emergence of novel mutated viruses causes a substantial dataset shift, which can greatly limit the performance of classification-based approaches. In this paper, we formulate the task of differentiating viral pneumonia from non-viral pneumonia and healthy controls into a one-class classification-based anomaly detection problem. We therefore propose the confidence-aware …

Go to article

Multiscale Generative Models: Improving Performance Of A Generative Model Using Feedback From Other Dependent Generative Models, Changyu Chen, Avinandan Bose, Shih-Fen Cheng, Arunesh Sinha Feb 2022

Multiscale Generative Models: Improving Performance Of A Generative Model Using Feedback From Other Dependent Generative Models, Changyu Chen, Avinandan Bose, Shih-Fen Cheng, Arunesh Sinha

Research Collection School Of Computing and Information Systems

Realistic fine-grained multi-agent simulation of real-world complex systems is crucial for many downstream tasks such as reinforcement learning. Recent work has used generative models (GANs in particular) for providing high-fidelity simulation of real-world systems. However, such generative models are often monolithic and miss out on modeling the interaction in multi-agent systems. In this work, we take a first step towards building multiple interacting generative models (GANs) that reflects the interaction in real world. We build and analyze a hierarchical set-up where a higher-level GAN is conditioned on the output of multiple lower-level GANs. We present a technique of using feedback …

Go to article

Choices Are Not Independent: Stackelberg Security Games With Nested Quantal Response Models, Tien Mai, Arunesh Sinha Feb 2022

Choices Are Not Independent: Stackelberg Security Games With Nested Quantal Response Models, Tien Mai, Arunesh Sinha

Research Collection School Of Computing and Information Systems

The quantal response (QR) model is widely used in Stackelberg security games (SSG) to model a bounded rational adversary. The QR model is a model of human response from among a large variety of prominent models known as discrete choice models. QR is the simplest type of discrete choice models and does not capture commonly observed phenomenon such as correlation among choices. We introduce the nested QR adversary model (based on nested logit model in discrete choice theory) in SSG which addresses shortcoming of the QR model. We present tractable approximation of the resulting equilibrium problem with nested QR adversary. …

Go to article

Field Study In Deploying Restless Multi-Armed Bandits: Assisting Non-Profits In Improving Maternal And Child Health, Aditya Mate, Lovish Madan, Aparna Taneja, Neha Madhiwalla, Shresth Verma, Gargi Singh, Aparna Hegde, Pradeep Varakantham, Milind Tambe Feb 2022

Field Study In Deploying Restless Multi-Armed Bandits: Assisting Non-Profits In Improving Maternal And Child Health, Aditya Mate, Lovish Madan, Aparna Taneja, Neha Madhiwalla, Shresth Verma, Gargi Singh, Aparna Hegde, Pradeep Varakantham, Milind Tambe

Research Collection School Of Computing and Information Systems

The widespread availability of cell phones has enabled non-profits to deliver critical health information to their beneficiaries in a timely manner. This paper describes our work to assist non-profits that employ automated messaging programs to deliver timely preventive care information to beneficiaries (new and expecting mothers) during pregnancy and after delivery. Unfortunately, a key challenge in such information delivery programs is that a significant fraction of beneficiaries drop out of the program. Yet, non-profits often have limited health-worker resources (time) to place crucial service calls for live interaction with beneficiaries to prevent such engagement drops. To assist non-profits in optimizing …

Go to article

Deep Graph-Level Anomaly Detection By Glocal Knowledge Distillation, Rongrong Ma, Guansong Pang, Ling Chen, Anton Van Den Hengel Feb 2022

Deep Graph-Level Anomaly Detection By Glocal Knowledge Distillation, Rongrong Ma, Guansong Pang, Ling Chen, Anton Van Den Hengel

Research Collection School Of Computing and Information Systems

Graph-level anomaly detection (GAD) describes the problem of detecting graphs that are abnormal in their structure and/or the features of their nodes, as compared to other graphs. One of the challenges in GAD is to devise graph representations that enable the detection of both locally- and globally-anomalous graphs, i.e., graphs that are abnormal in their fine-grained (node-level) or holistic (graph-level) properties, respectively. To tackle this challenge we introduce a novel deep anomaly detection approach for GAD that learns rich global and local normal pattern information by joint random distillation of graph and node representations. The random distillation is achieved by …

Go to article

Perceptions And Needs Of Artificial Intelligence In Health Care To Increase Adoption: Scoping Review, Han Shi Jocelyn Chew, Palakorn Achananuparp Jan 2022

Perceptions And Needs Of Artificial Intelligence In Health Care To Increase Adoption: Scoping Review, Han Shi Jocelyn Chew, Palakorn Achananuparp

Research Collection School Of Computing and Information Systems

Background: Artificial intelligence (AI) has the potential to improve the efficiency and effectiveness of health care service delivery. However, the perceptions and needs of such systems remain elusive, hindering efforts to promote AI adoption in health care. Objective: This study aims to provide an overview of the perceptions and needs of AI to increase its adoption in health care. Methods: A systematic scoping review was conducted according to the 5-stage framework by Arksey and O’Malley. Articles that described the perceptions and needs of AI in health care were searched across nine databases: ACM Library, CINAHL, Cochrane Central, Embase, IEEE Xplore, …

Go to article

Beyond Triplet Loss: Person Re-Identification With Fine-Grained Difference-Aware Pairwise Loss, Cheng Yan, Guansong Pang, Xiao Bai, Changhong Liu, Xin Ning, Jun Zhou Jan 2022

Beyond Triplet Loss: Person Re-Identification With Fine-Grained Difference-Aware Pairwise Loss, Cheng Yan, Guansong Pang, Xiao Bai, Changhong Liu, Xin Ning, Jun Zhou

Research Collection School Of Computing and Information Systems

Person Re-IDentification (ReID) aims at re-identifying persons from different viewpoints across multiple cameras. Capturing the fine-grained appearance differences is often the key to accurate person ReID, because many identities can be differentiated only when looking into these fine-grained differences. However, most state-of-the-art person ReID approaches, typically driven by a triplet loss, fail to effectively learn the fine-grained features as they are focused more on differentiating large appearance differences. To address this issue, we introduce a novel pairwise loss function that enables ReID models to learn the fine-grained features by adaptively enforcing an exponential penalization on the images of small differences …

Go to article

Cross-Modal Food Retrieval: Learning A Joint Embedding Of Food Images And Recipes With Semantic Consistency And Attention Mechanism, Hao Wang, Doyen Sahoo, Chenghao Liu, Ke Shu, Palakorn Achananuparp, Ee-Peng Lim, Steven C. H. Hoi Jan 2022

Cross-Modal Food Retrieval: Learning A Joint Embedding Of Food Images And Recipes With Semantic Consistency And Attention Mechanism, Hao Wang, Doyen Sahoo, Chenghao Liu, Ke Shu, Palakorn Achananuparp, Ee-Peng Lim, Steven C. H. Hoi

Research Collection School Of Computing and Information Systems

Food retrieval is an important task to perform analysis of food-related information, where we are interested in retrieving relevant information about the queried food item such as ingredients, cooking instructions, etc. In this paper, we investigate cross-modal retrieval between food images and cooking recipes. The goal is to learn an embedding of images and recipes in a common feature space, such that the corresponding image-recipe embeddings lie close to one another. Two major challenges in addressing this problem are 1) large intra-variance and small inter-variance across cross-modal food data; and 2) difficulties in obtaining discriminative recipe representations. To address these …

Go to article

Adadeep: A Usage-Driven, Automated Deep Model Compression Framework For Enabling Ubiquitous Intelligent Mobiles, Sicong Liu, Junzhao Du, Kaiming Nan, Zimu Zhou, Hui Liu, Zhangyang Wang, Yingyan Lin Dec 2021

Adadeep: A Usage-Driven, Automated Deep Model Compression Framework For Enabling Ubiquitous Intelligent Mobiles, Sicong Liu, Junzhao Du, Kaiming Nan, Zimu Zhou, Hui Liu, Zhangyang Wang, Yingyan Lin

Research Collection School Of Computing and Information Systems

Recent breakthroughs in deep neural networks (DNNs) have fueled a tremendously growing demand for bringing DNN-powered intelligence into mobile platforms. While the potential of deploying DNNs on resource-constrained platforms has been demonstrated by DNN compression techniques, the current practice suffers from two limitations: 1) merely stand-alone compression schemes are investigated even though each compression technique only suit for certain types of DNN layers; and 2) mostly compression techniques are optimized for DNNs’ inference accuracy, without explicitly considering other application-driven system performance (e.g., latency and energy cost) and the varying resource availability across platforms (e.g., storage and processing capability). To this …

Go to article

Ai And The Future Of Work: What We Know Today, Steven M. Miller, Thomas H. Davenport Dec 2021

Ai And The Future Of Work: What We Know Today, Steven M. Miller, Thomas H. Davenport

Research Collection School Of Computing and Information Systems

To contribute to a better understanding of the contemporary realities of AI workplace deployments, the authors recently completed 29 case studies of people doing their everyday work with AI-enabled smart machines. Twenty-three of these examples were from North America, mostly in the US. Six were from Southeast Asia, mostly in Singapore. In this essay, we compare our findings on job and workplace impacts to those reported in the MIT Task Force on the Work of the Future report, as we consider that to be the most comprehensive recent study on this topic.

Go to article

Solving The Vehicle Routing Problem With Simultaneous Pickup And Delivery And Occasional Drivers By Simulated Annealing, Vincent F. Yu, Grace Aloina, Panca Jodiawan, Aldy Gunawan, Tsung-Chi Huang Dec 2021

Solving The Vehicle Routing Problem With Simultaneous Pickup And Delivery And Occasional Drivers By Simulated Annealing, Vincent F. Yu, Grace Aloina, Panca Jodiawan, Aldy Gunawan, Tsung-Chi Huang

Research Collection School Of Computing and Information Systems

This research studies the vehicle routing problem with simultaneous pickup and delivery with an occasional driver (VRPSPDOD). VRPSPDOD is a new variant of the vehicle routing problems with simultaneous pickup and delivery (VRPSPD). Different from VRPSPD, in VRPSPDOD, occasional drivers are employed to work with regular vehicles to service customers’ pickup and delivery requests in order to minimize the total cost. We formulate a mixed integer linear programming model for VRPSPD and propose a heuristic algorithm based on simulated annealing (SA) to solve the problem. The results of comprehensive numerical experiments show that the proposed SA performs well in terms …

Go to article

Building Action Sets In A Deep Reinforcement Learner, Yongzhao Wang, Arunesh Sinha, Sky C.H. Wang, Michael P. Wellman Dec 2021

Building Action Sets In A Deep Reinforcement Learner, Yongzhao Wang, Arunesh Sinha, Sky C.H. Wang, Michael P. Wellman

Research Collection School Of Computing and Information Systems

In many policy-learning applications, the agent may execute a set of actions at each decision stage. Choosing among an exponential number of alternatives poses a computational challenge, and even representing actions naturally expressed as sets can be a tricky design problem. Building upon prior approaches that employ deep neural networks and iterative construction of action sets, we introduce a reward-shaping approach to apportion reward to each atomic action based on its marginal contribution within an action set, thereby providing useful feedback for learning to build these sets. We demonstrate our method in two environments where action spaces are combinatorial. Experiments …

Go to article

Rmm: Reinforced Memory Management For Class-Incremental Learning, Yaoyao Liu, Qianru Sun, Qianru Sun Dec 2021

Rmm: Reinforced Memory Management For Class-Incremental Learning, Yaoyao Liu, Qianru Sun, Qianru Sun

Research Collection School Of Computing and Information Systems

Class-Incremental Learning (CIL) [38] trains classifiers under a strict memory budget: in each incremental phase, learning is done for new data, most of which is abandoned to free space for the next phase. The preserved data are exemplars used for replaying. However, existing methods use a static and ad hoc strategy for memory allocation, which is often sub-optimal. In this work, we propose a dynamic memory management strategy that is optimized for the incremental phases and different object classes. We call our method reinforced memory management (RMM), leveraging reinforcement learning. RMM training is not naturally compatible with CIL as the …

Go to article

Rmix: Learning Risk-Sensitive Policies For Cooperative Reinforcement Learning Agents, Wei Qiu, Xinrun Wang, Runsheng Yu, Xu He, Rundong Wang, Bo An, Svetlana Obraztsova, Zinovi Rabinovich Dec 2021

Rmix: Learning Risk-Sensitive Policies For Cooperative Reinforcement Learning Agents, Wei Qiu, Xinrun Wang, Runsheng Yu, Xu He, Rundong Wang, Bo An, Svetlana Obraztsova, Zinovi Rabinovich

Research Collection School Of Computing and Information Systems

Current value-based multi-agent reinforcement learning methods optimize individual Q values to guide individuals' behaviours via centralized training with decentralized execution (CTDE). However, such expected, i.e., risk-neutral, Q value is not sufficient even with CTDE due to the randomness of rewards and the uncertainty in environments, which causes the failure of these methods to train coordinating agents in complex environments. To address these issues, we propose RMIX, a novel cooperative MARL method with the Conditional Value at Risk (CVaR) measure over the learned distributions of individuals' Q values. Specifically, we first learn the return distributions of individuals to analytically calculate CVaR …

Go to article

Hierarchical Control Of Multi-Agent Reinforcement Learning Team In Real-Time Strategy (Rts) Games, Weigui Jair Zhou, Budhitama Subagdja, Ah-Hwee Tan, Darren Wee Sze Ong Dec 2021

Hierarchical Control Of Multi-Agent Reinforcement Learning Team In Real-Time Strategy (Rts) Games, Weigui Jair Zhou, Budhitama Subagdja, Ah-Hwee Tan, Darren Wee Sze Ong

Research Collection School Of Computing and Information Systems

Coordinated control of multi-agent teams is an important task in many real-time strategy (RTS) games. In most prior work, micromanagement is the commonly used strategy whereby individual agents operate independently and make their own combat decisions. On the other extreme, some employ a macromanagement strategy whereby all agents are controlled by a single decision model. In this paper, we propose a hierarchical command and control architecture, consisting of a single high-level and multiple low-level reinforcement learning agents operating in a dynamic environment. This hierarchical model enables the low-level unit agents to make individual decisions while taking commands from the high-level …

Go to article

Learning Knowledge-Enriched Company Embeddings For Investment Management, Gary Ang, Ee-Peng Lim Nov 2021

Learning Knowledge-Enriched Company Embeddings For Investment Management, Gary Ang, Ee-Peng Lim

Research Collection School Of Computing and Information Systems

Relationships between companies serve as key channels through which the effects of past stock price movements and news events propagate and influence future price movements. Such relationships can be implicitly found in knowledge bases or explicitly represented as knowledge graphs. In this paper, we propose KnowledgeEnriched Company Embedding (KECE), a novel multi-stage attentionbased dynamic network embedding model combining multimodal information of companies with knowledge from Wikipedia and knowledge graph relationships from Wikidata to generate company entity embeddings that can be applied to a variety of downstream investment management tasks. Experiments on an extensive set of real-world stock prices and news …

Go to article

Span-Level Emotion Cause Analysis With Neural Sequence Tagging, Xiangju Li, Wei Gao, Shi Feng, Daling Wang, Shafiq Joty Nov 2021

Span-Level Emotion Cause Analysis With Neural Sequence Tagging, Xiangju Li, Wei Gao, Shi Feng, Daling Wang, Shafiq Joty

Research Collection School Of Computing and Information Systems

This paper addresses the task of span-level emotion cause analysis (SECA). It is a finer-grained emotion cause analysis (ECA) task, which aims to identify the specific emotion cause span(s) behind certain emotions in text. In this paper, we formalize SECA as a sequence tagging task for which several variants of neural network-based sequence tagging models to extract specific emotion cause span(s) in the given context. These models combine different types of encoding and decoding approaches. Furthermore, to make our models more "emotionally sensitive'', we utilize the multi-head attention mechanism to enhance the representation of context. Experimental evaluations conducted on two …

Go to article

First Prev Next Last