Artificial Intelligence and Robotics Commons^™

Creating Data From Unstructured Text With Context Rule Assisted Machine Learning (Craml), Stephen Meisenbacher, Peter Norlander

School of Business: Faculty Publications and Other Works

Popular approaches to building data from unstructured text come with limitations, such as scalability, interpretability, replicability, and real-world applicability. These can be overcome with Context Rule Assisted Machine Learning (CRAML), a method and no-code suite of software tools that builds structured, labeled datasets which are accurate and reproducible. CRAML enables domain experts to access uncommon constructs within a document corpus in a low-resource, transparent, and flexible manner. CRAML produces document-level datasets for quantitative research and makes qualitative classification schemes scalable over large volumes of text. We demonstrate that the method is useful for bibliographic analysis, transparent analysis of proprietary data, …

Conversation Disentanglement With Bi-Level Contrastive Learning, Chengyu Huang, Zheng Zhang, Hao Fei, Lizi Liao

Research Collection School Of Computing and Information Systems

Conversation disentanglement aims to group utterances into detached sessions, which is a fundamental task in processing multi-party conversations. Existing methods have two main drawbacks. First, they overemphasize pairwise utterance relations but pay inadequate attention to the utterance-to-context relation modeling. Second, a huge amount of human annotated data is required for training, which is expensive to obtain in practice. To address these issues, we propose a general disentangle model based on bi-level contrastive learning. It brings closer utterances in the same session while encourages each utterance to be near its clustered session prototypes in the representation space. Unlike existing approaches, our …

End-To-End Hierarchical Reinforcement Learning With Integrated Subgoal Discovery, Shubham Pateria, Budhitama Subagdja, Ah-Hwee Tan, Chai Quek

Research Collection School Of Computing and Information Systems

Hierarchical reinforcement learning (HRL) is a promising approach to perform long-horizon goal-reaching tasks by decomposing the goals into subgoals. In a holistic HRL paradigm, an agent must autonomously discover such subgoals and also learn a hierarchy of policies that uses them to reach the goals. Recently introduced end-to-end HRL methods accomplish this by using the higher-level policy in the hierarchy to directly search the useful subgoals in a continuous subgoal space. However, learning such a policy may be challenging when the subgoal space is large. We propose integrated discovery of salient subgoals (LIDOSS), an end-to-end HRL method with an integrated …

S-Prompts Learning With Pre-Trained Transformers: An Occam's Razor For Domain Incremental Learning, Yabin Wang, Zhiwu Huang, Xiaopeng. Hong

Research Collection School Of Computing and Information Systems

State-of-the-art deep neural networks are still struggling to address the catastrophic forgetting problem in continual learning. In this paper, we propose one simple paradigm (named as S-Prompting) and two concrete approaches to highly reduce the forgetting degree in one of the most typical continual learning scenarios, i.e., domain increment learning (DIL). The key idea of the paradigm is to learn prompts independently across domains with pre-trained transformers, avoiding the use of exemplars that commonly appear in conventional methods. This results in a win-win game where the prompting can achieve the best for each domain. The independent prompting across domains only …

Prompting For Multimodal Hateful Meme Classification, Rui Cao, Roy Ka-Wei Lee, Wen-Haw Chong, Jing Jiang

Research Collection School Of Computing and Information Systems

Hateful meme classification is a challenging multimodal task that requires complex reasoning and contextual background knowledge. Ideally, we could leverage an explicit external knowledge base to supplement contextual and cultural information in hateful memes. However, there is no known explicit external knowledge base that could provide such hate speech contextual information. To address this gap, we propose PromptHate, a simple yet effective prompt-based model that prompts pre-trained language models (PLMs) for hateful meme classification. Specifically, we construct simple prompts and provide a few in-context examples to exploit the implicit knowledge in the pretrained RoBERTa language model for hateful meme classification. …

A Unified Dialogue User Simulator For Few-Shot Data Augmentation, Dazhen Wan, Zheng Zhang, Qi Zhu, Lizi Liao, Minlie Huang

Research Collection School Of Computing and Information Systems

Pre-trained language models have shown superior performance in task-oriented dialogues. However, existing datasets are on limited scales, which cannot support large-scale pre-training. Fortunately, various data augmentation methods have been developed to augment largescale task-oriented dialogue corpora. However, they heavily rely on annotated data in the target domain, which require a tremendous amount of data collection and human labeling work. In this paper, we build a unified dialogue user simulation model by pre-training on several publicly available datasets. The model can then be tuned on a target domain with fewshot data. The experiments on a target dataset across multiple domains show …

Photovoltaic Cells For Energy Harvesting And Indoor Positioning, Hamada Rizk, Dong Ma, Mahbub Hassan, Moustafa Youssef

Research Collection School Of Computing and Information Systems

We propose SoLoc, a lightweight probabilistic fingerprinting-based technique for energy-free device-free indoor localization. The system harnesses photovoltaic currents harvested by the photovoltaic cells in smart environments for simultaneously powering digital devices and user positioning. The basic principle is that the location of the human interferes with the lighting received by the photovoltaic cells, thus producing a location fingerprint on the generated photocurrents. To ensure resilience to noisy measurements, SoLoc constructs probability distributions as a photovoltaic fingerprint at each location. Then, we employ a probabilistic graphical model for estimating the user location in the continuous space. Results show that SoLoc can …

Artificial Intelligence For Natural Disaster Management, Guansong Pang

Research Collection School Of Computing and Information Systems

Artificial intelligence (AI) can leverage massive amount of diverse types of data, such as geospatial data, social media data, and wireless network sensor data, to enhance our understanding of natural disasters, their forecasting and detection, and humanitarian assistance in natural disaster management (NDM). Due to this potential, different communities have been dedicating enormous efforts to the development and/or adoption of AI technologies for NDM. This article provides an overview of these efforts and discusses major challenges and opportunities in this topic.

Neural Approaches For Language-Agnostic Search And Recommendation, Hamed Rezanejad Asl Bonab

Doctoral Dissertations

There are significant efforts toward developing better neural approaches for information retrieval problems. However, the vast majority of these studies are conducted using English-only data. In fact, trends and statistics of non-English content and users on the Internet show exponential growth and that novel information retrieval systems need to be language-agnostic; they need to bridge the language barrier between users and content, leverage data from high-resource settings for lower-resourced settings, and be able to extend to new languages and local markets easily. To this end, we focus on search and recommendation as two vital components of information systems. We explore …

Multi-Functional Job Roles To Support Operations In A Multi-Faceted Jewel Enabled By Ai And Digital Transformation, Steven M. Miller

Research Collection School Of Computing and Information Systems

In this story, we highlight the way in which the use of AI enabled support systems, together with work process digital transformation and innovative approaches to job redesign, have combined to dramatically change the nature of the work of the front-line service staff who protect and support the facility and visitors at the world’s most iconic airport mall and lifestyle destination.

Artificial Intelligence, Consumers, And The Experience Economy, Hannah H. Chang, Anirban Mukherjee

Research Collection Lee Kong Chian School Of Business

The term Artificial Intelligence (AI) was first used by McCarthy, Minsky, Rochester, and Shannon in a proposal for a summer research project in 1955 (Solomonoff, 1985). It is widely and commonly defined to be “the science and engineering of making intelligent machines” (McCarthy, 2006). Recent technological advances and methodological developments have made AI pervasive in new marketing offerings, ranging from self-driving cars, intelligent voice assistants such as Amazon’s Alexa, to burger-making robots at restaurants and rack-moving robots inside warehouses such as Amazon’s family of robots (Kiva, Pegasus, Xanthus) and delivery drones. There is optimism, and perhaps even over-optimism, of the …

Two Singapore Public Healthcare Ai Applications For National Screening Programs And Other Examples, Andy Wee An Ta, Han Leong Goh, Christine Ang, Lian Yeow Koh, Ken Poon, Steven M. Miller

Research Collection School Of Computing and Information Systems

This article explains how two AI systems have been incorporated into the everyday operations of two Singapore public healthcare nation-wide screening programs. The first example is embedded within the setting of a national level population health screening program for diabetes related eye diseases, targeting the rapidly increasing number of adults in the country with diabetes. In the second example, the AI assisted screening is done shortly after a person is admitted to one of the public hospitals to identify which inpatients—especially which elderly patients with complex conditions—have a high risk of being readmitted as an inpatient multiple times in the …

Constrained Multiagent Reinforcement Learning For Large Agent Population, Jiajing Ling, Arambam James Singh, Duc Thien Nguyen, Akshat Kumar

Research Collection School Of Computing and Information Systems

Learning control policies for a large number of agents in a decentralized setting is challenging due to partial observability, uncertainty in the environment, and scalability challenges. While several scalable multiagent RL (MARL) methods have been proposed, relatively few approaches exist for large scale constrained MARL settings. To address this, we first formulate the constrained MARL problem in a collective multiagent setting where interactions among agents are governed by the aggregate count and types of agents, and do not depend on agents’ specific identities. Second, we show that standard Lagrangian relaxation methods, which are popular for single agent RL, do not …

Deep Learning-Based Text Recognition Of Agricultural Regulatory Document, Hua Leong Fwa, Farn Haur Chan

Research Collection School Of Computing and Information Systems

In this study, an OCR system based on deep learning techniques was deployed to digitize scanned agricultural regulatory documents comprising of certificates and labels. Recognition of the certificates and labels is challenging as they are scanned images of the hard copy form and the layout and size of the text as well as the languages vary between the various countries (due to diverse regulatory requirements). We evaluated and compared between various state-of-the-art deep learningbased text detection and recognition model as well as a packaged OCR library – Tesseract. We then adopted a two-stage approach comprising of text detection using Character …

Improving Deep Entity Resolution By Constraints, Soudeh Nilforoushan

Electronic Thesis and Dissertation Repository

Entity resolutions the problem of finding duplicate data in a dataset and resolving possible differences and inconsistencies. ER is a long-standing data management and information retrieval problem and a core data integration and cleaning task. There are diverse solutions for ER that apply rule-based techniques, pairwise binary classification, clustering, and probabilistic inference, among other techniques. Deep learning (DL) has been extensively used for ER and has shown competitive performance compared to conventional ER solutions. The state-of-the-art (SOTA) ER solutions using DL are based on pairwise comparison and binary classification. They transform pairs of records into a latent space that can …

Trajectory Optimization For Safe Navigation In Maritime Traffic Using Historical Data, Chaithanya Basrur, Arambam James Singh, Arunesh Sinha, Akshat Kumar, T. K. Satish Kumar

Research Collection School Of Computing and Information Systems

Increasing maritime trade often results in congestion in busy ports, thereby necessitating planning methods to avoid close quarter risky situations among vessels. Rapid digitization and automation of port operations and vessel navigation provide unique opportunities for significantly improving navigation safety. Our key contributions are as follows. First, given a set of future candidate trajectories for vessels in a traffic hotspot zone, we develop a multiagent trajectory optimization method to choose trajectories that result in the best overall close quarter risk reduction. Our novel MILP-based optimization method is more than an order-of-magnitude faster than a standard MILP for this problem, and …

Reputation-Based Trust Assessment Of Transacting Service Components, Konstantinos Tsiounis

Electronic Thesis and Dissertation Repository

As Service-Oriented Systems rely for their operation on many different, and most often, distributed software components, a key issue that emerges is how one component can trust the services offered by another component. Here, the concept of trust is considered in the context of reputation systems and is viewed as a meta-requirement, that is, the level of belief a service requestor has that a service provider will provide the service in a way that meets the requestor’s expectations. We refer to the service offering components as service providers (SPs) and the service requesting components as service clients (SCs).

In this …

Ai-Enabled Adaptive Learning Using Automated Topic Alignment And Doubt Detection, Kar Way Tan, Siaw Ling Lo, Eng Lieh Ouh, Wei Leng Neo

Research Collection School Of Computing and Information Systems

Implementing adaptive learning is often a challenging task at higher learning institutions where the students come from diverse backgrounds and disciplines. In this work, we collected informal learning journals from learners. Using the journals, we trained two machine learning models, an automated topic alignment and a doubt detection model to identify areas of adjustment required for teaching and students who require additional attention. The models form the baseline for a quiz recommender tool to dynamically generate personalized quizzes for each learner as practices to reinforce learning. Our pilot deployment of our AI-enabled Adaptive Learning System showed that our approach delivers …

A Mean-Field Markov Decision Process Model For Spatial Temporal Subsidies In Ride-Sourcing Markets, Zheng Zhu, Jintao Ke, Hai Wang

Research Collection School Of Computing and Information Systems

Ride-sourcing services are increasingly popular because of their ability to accommodate on-demand travel needs. A critical issue faced by ride-sourcing platforms is the supply-demand imbalance, as a result of which drivers may spend substantial time on idle cruising and picking up remote passengers. Some platforms attempt to mitigate the imbalance by providing relocation guidance for idle drivers who may have their own self-relocation strategies and decline to follow the suggestions. Platforms then seek to induce drivers to system-desirable locations by offering them subsidies. This paper proposes a mean-field Markov decision process (MF-MDP) model to depict the dynamics in ride-sourcing markets …

What Makes The Story Forward?: Inferring Commonsense Explanations As Prompts For Future Event Generation, Li Lin, Yixin Cao, Lifu Huang, Shu Ang Li, Xuming Hu, Lijie Wen, Jianmin Wang

Research Collection School Of Computing and Information Systems

Prediction over event sequences is critical for many real-world applications in Information Retrieval and Natural Language Processing. Future Event Generation (FEG) is a challenging task in event sequence prediction because it requires not only fluent text generation but also commonsense reasoning to maintain the logical coherence of the entire event story. In this paper, we propose a novel explainable FEG framework, Coep. It highlights and integrates two types of event knowledge, sequential knowledge of direct event-event relations and inferential knowledge that reflects the intermediate character psychology between events, such as intents, causes, reactions, which intrinsically pushes the story forward. To …

Test Mimicry To Assess The Exploitability Of Library Vulnerabilities, Hong Jin Kang, Truong Giang Nguyen, Bach Le, Corina S. Pasareanu, David Lo

Research Collection School Of Computing and Information Systems

Modern software engineering projects often depend on open-source software libraries, rendering them vulnerable to potential security issues in these libraries. Developers of client projects have to stay alert of security threats in the software dependencies. While there are existing tools that allow developers to assess if a library vulnerability is reachable from a project, they face limitations. Call graphonly approaches may produce false alarms as the client project may not use the vulnerable code in a way that triggers the vulnerability, while test generation-based approaches faces difficulties in overcoming the intrinsic complexity of exploiting a vulnerability, where extensive domain knowledge …

Cosm2ic: Optimizing Real-Time Multi-Modal Instruction Comprehension, Weerakoon Mudiyanselage Dulanga Kaveesha Weerakoon, Vigneshwaran Subbaraju, Minh Anh Tuan Tran, Archan Misra

Research Collection School Of Computing and Information Systems

Supporting real-time, on-device execution of multi-modal referring instruction comprehension models is an important challenge to be tackled in embodied Human-Robot Interaction. However, state-of-the-art deep learning models are resource-intensive and unsuitable for real-time execution on embedded devices. While model compression can achieve a reduction in computational resources up to a certain point, further optimizations result in a severe drop in accuracy. To minimize this loss in accuracy, we propose the COSM2IC framework, with a lightweight Task Complexity Predictor, that uses multiple sensor inputs to assess the instructional complexity and thereby dynamically switch between a set of models of varying computational intensity …

Structured And Natural Responses Co-Generation For Conversational Search, Chenchen Ye, Lizi Liao, Fuli Feng, Wei Ji, Tat-Seng Chua

Research Collection School Of Computing and Information Systems

Generating fluent and informative natural responses while maintaining representative internal states for search optimization is critical for conversational search systems. Existing approaches either 1) predict structured dialog acts first and then generate natural response; or 2) map conversation context to natural responses directly in an end-to-end manner. Both kinds of approaches have shortcomings. The former suffers from error accumulation while the semantic associations between structured acts and natural responses are confined in single direction. The latter emphasizes generating natural responses but fails to predict structured acts. Therefore, we propose a neural co-generation model that generates the two concurrently. The key …

Multi-Level Cross-View Contrastive Learning For Knowledge-Aware Recommender System, Ding Zou, Wei Wei, Xian-Ling Mao, Ziyang Wang, Minghui Qiu, Feida Zhu, Xin Cao

Research Collection School Of Computing and Information Systems

Knowledge graph (KG) plays an increasingly important role in recommender systems. Recently, graph neural networks (GNNs) based model has gradually become the theme of knowledge-aware recommendation (KGR). However, there is a natural deficiency for GNN-based KGR models, that is, the sparse supervised signal problem, which may make their actual performance drop to some extent. Inspired by the recent success of contrastive learning in mining supervised signals from data itself, in this paper, we focus on exploring the contrastive learning in KG-aware recommendation and propose a novel multi-level cross-view contrastive learning mechanism, named MCCLK. Different from traditional contrastive learning methods which …

Towards Aligning Slides And Video Snippets: Mitigating Sequence And Content Mismatches, Ziyuan Liu, Hady W. Lauw

Research Collection School Of Computing and Information Systems

Slides are important form of teaching materials used in various courses at academic institutions. Due to their compactness, slides on their own may not stand as complete reference materials. To aid students’ understanding, it would be useful to supplement slides with other materials such as online videos. Given a deck of slides and a related video, we seek to align each slide in the deck to a relevant video snippet, if any. While this problem could be formulated as aligning two time series (each involving a sequence of text contents), we anticipate challenges in generating matches arising from differences in …

Using Constraint Programming And Graph Representation Learning For Generating Interpretable Cloud Security Policies, Mikhail Kazdagli, Mohit Tiwari, Akshat Kumar

Research Collection School Of Computing and Information Systems

Modern software systems rely on mining insights from business sensitive data stored in public clouds. A data breach usually incurs signifcant (monetary) loss for a commercial organization. Conceptually, cloud security heavily relies on Identity Access Management (IAM) policies that IT admins need to properly confgure and periodically update. Security negligence and human errors often lead to misconfguring IAM policies which may open a backdoor for attackers. To address these challenges, frst, we develop a novel framework that encodes generating optimal IAM policies using constraint programming (CP). We identify reducing dormant permissions of cloud users as an optimality criterion, which intuitively …

Nonparametric Contextual Reasoning For Question Answering Over Large Knowledge Bases, Rajarshi Das

Doctoral Dissertations

Question answering (QA) over knowledge bases provides a user-friendly way of accessing the massive amount of information stored in them. We have experienced tremendous progress in the performance of QA systems, thanks to the recent advancements in representation learning by deep neural models. However, such deep models function as black boxes with an opaque reasoning process, are brittle, and offer very limited control (e.g. for debugging an erroneous model prediction). It is also unclear how to reliably add or update knowledge stored in their model parameters. This thesis proposes nonparametric models for question answering that disentangle logic from knowledge. For …

Multimodal Zero-Shot Hateful Meme Detection, Jiawen Zhu, Roy Ka-Wei Lee, Wen Haw Chong

Research Collection School Of Computing and Information Systems

Facebook has recently launched the hateful meme detection challenge, which garnered much attention in academic and industry research communities. Researchers have proposed multimodal deep learning classification methods to perform hateful meme detection. While the proposed methods have yielded promising results, these classification methods are mostly supervised and heavily rely on labeled data that are not always available in the real-world setting. Therefore, this paper explores and aims to perform hateful meme detection in a zero-shot setting. Working towards this goal, we propose Target-Aware Multimodal Enhancement (TAME), which is a novel deep generative framework that can improve existing hateful meme classification …

Simultaneous Energy Harvesting And Gait Recognition Using Piezoelectric Energy Harvester, Dong Ma, Guohao Lan, Weitao Xu, Mahbub Hassan, Wen Hu

Research Collection School Of Computing and Information Systems

Piezoelectric energy harvester, which generates electricity from stress or vibrations, is gaining increasing attention as a viable solution to extend battery life in wearables. Recent research further reveals that, besides generating energy, PEH can also serve as a passive sensor to detect human gait power-efficiently because its stress or vibration patterns are significantly influenced by the gait. However, as PEHs are not designed for precise measurement of motion, achievable gait recognition accuracy remains low with conventional classification algorithms. The accuracy deteriorates further when the generated electricity is stored simultaneously. To classify gait reliably while simultaneously storing generated energy, we make …

Generative Flows With Invertible Attentions, Rhea Sanjay Sukthanker, Zhiwu Huang, Suryansh Kumar, Radu Timofte, Luc Van Gool

Research Collection School Of Computing and Information Systems

Flow-based generative models have shown an excellent ability to explicitly learn the probability density function of data via a sequence of invertible transformations. Yet, learning attentions in generative flows remains understudied, while it has made breakthroughs in other domains. To fill the gap, this paper introduces two types of invertible attention mechanisms, i.e., map-based and transformer-based attentions, for both unconditional and conditional generative flows. The key idea is to exploit a masked scheme of these two attentions to learn long-range data dependencies in the context of generative flows. The masked scheme allows for invertible attention modules with tractable Jacobian determinants, …