Artificial Intelligence and Robotics Commons^™

Transformers In Remote Sensing: A Survey, Abdulaziz Amer Aleissaee, Amandeep Kumar, Rao Anwer, Salman Khan, Hisham Cholakkal, Gui-Song Xia, Fahad Shahbaz Khan

Computer Vision Faculty Publications

Deep learning-based algorithms have seen a massive popularity in different areas of remote sensing image analysis over the past decade. Recently, transformers-based architectures, originally introduced in natural language processing, have pervaded computer vision field where the self-attention mechanism has been utilized as a replacement to the popular convolution operator for capturing long-range dependencies. Inspired by recent advances in computer vision, remote sensing community has also witnessed an increased exploration of vision transformers for a diverse set of tasks. Although a number of surveys have focused on transformers in computer vision in general, to the best of our knowledge we are …

Accomontage2: A Complete Harmonization And Accompaniment Arrangement System, Li Yi, Haochen Hu, Jingwei Zhao, Gus Xia

Machine Learning Faculty Publications

We propose AccoMontage2, a system capable of doing full-length song harmonization and accompaniment arrangement based on a lead melody. Following AccoMontage, this study focuses on generating piano arrangements for popular/folk songs and it carries on the generalized template-based retrieval method. The novelties of this study are twofold. First, we invent a harmonization module (which AccoMontage does not have). This module generates structured and coherent full-length chord progression by optimizing and balancing three loss terms: a micro-level loss for note-wise dissonance, a meso-level loss for phrase-template matching, and a macro-level loss for full piece coherency. Second, we develop a graphical user …

Overview Of The Clef-2022 Checkthat! Lab Task 2 On Detecting Previously Fact-Checked Claims, Preslav Nakov, Giovanni Da San Martino, Firoj Alam, Shaden Shaar, Hamdy Mubarak, Nikolay Babulkov

Natural Language Processing Faculty Publications

We describe the fourth edition of the CheckThat! Lab, part of the 2022 Conference and Labs of the Evaluation Forum (CLEF). The lab evaluates technology supporting three tasks related to factuality, and it covers seven languages such as Arabic, Bulgarian, Dutch, English, German, Spanish, and Turkish. Here, we present the task 2, which asks to detect previously fact-checked claims (in two languages). A total of six teams participated in this task, submitted a total of 37 runs, and most submissions managed to achieve sizable improvements over the baselines using transformer based models such as BERT, RoBERTa. In this paper, we …

Negational Symmetry Of Quantum Neural Networks For Binary Pattern Classification, Nanqing Dong, Michael Kampffmeyer, Irina Voiculescu, Eric P. Xing

Machine Learning Faculty Publications

Although quantum neural networks (QNNs) have shown promising results in solving simple machine learning tasks recently, the behavior of QNNs in binary pattern classification is still underexplored. In this work, we find that QNNs have an Achilles’ heel in binary pattern classification. To illustrate this point, we provide a theoretical insight into the properties of QNNs by presenting and analyzing a new form of symmetry embedded in a family of QNNs with full entanglement, which we term negational symmetry. Due to negational symmetry, QNNs can not differentiate between a quantum binary signal and its negational counterpart. We empirically evaluate the …

Overview Of The Clef-2022 Checkthat! Lab Task 1 On Identifying Relevant Claims In Tweets, Preslav Nakov, Alberto Barrón-Cedeño, Giovanni Da San Martino, Firoj Alam, Mucahid Kutlu, Wajdi Zaghouani, Mucahid Kutlu, Wajdi Zaghouani, Chengkai Li, Shaden Shaar, Hamdy Mubarak, Alex Nikolov

Natural Language Processing Faculty Publications

We present an overview of CheckThat! lab 2022 Task 1, part of the 2022 Conference and Labs of the Evaluation Forum (CLEF). Task 1 asked to predict which posts in a Twitter stream are worth fact-checking, focusing on COVID-19 and politics in six languages: Arabic, Bulgarian, Dutch, English, Spanish, and Turkish. A total of 19 teams participated and most submissions managed to achieve sizable improvements over the baselines using Transformer-based models such as BERT and GPT-3. Across the four subtasks, approaches that targetted multiple languages (be it individually or in conjunction, in general obtained the best performance. We describe the …

Artificial Intelligence And Human Employment, Singapore Management University

Perspectives@SMU

AI will replace humans in repetitive tasks. Greater value can be created when it augments and complements the jobs people do

Products Pricing And Return Strategies For The Dual Channel Retailers, Jian Liu, Xinyue Sun, Yanyan Liu

Electrical and Computer Engineering Faculty Research & Creative Works

This paper analyzed how different return strategies and return rates affect dual-channel retailers' profits and channel pricings. Return can stimulate sales; however, the return has presented significant challenges to retailers. The return has long been studied to maximize profit and pricing; however, the different return strategies for dual-channel retailers affect channel both. This paper aimed to study whether or not dual-channel retailers should allow customers to return items in two channels and whether or not the retailer should contract with the manufacturers and pay extra fees to return products. This study indicated when the retailer should allow customers' returns to …

Leveraging Subject Matter Expertise To Optimize Machine Learning Techniques For Air And Space Applications, Philip Y. Cho

Theses and Dissertations

We develop new machine learning and statistical methods that are tailored for Air and Space applications through the incorporation of subject matter expertise. In particular, we focus on three separate research thrusts that each represents a different type of subject matter knowledge, modeling approach, and application. In our first thrust, we incorporate knowledge of natural phenomena to design a neural network algorithm for localizing point defects in transmission electron microscopy (TEM) images of crystalline materials. In our second research thrust, we use Bayesian feature selection and regression to analyze the relationship between fighter pilot attributes and flight mishap rates. We …

Degrees Of Confidence As A Legal Tool To Assess Ai System Liability, Joshua Song

Michigan Technology Law Review

AI systems have become increasingly integrated into our everyday lives, and harms caused by these systems have graduated from raising hypothetical ethical concerns to questions of actual legal liability. Civil liability schemes are generally designed to address harms caused by humans; thus, it may be tempting to analogize new types of harms caused by AI systems to familiar harms caused by humans in order to justify commandeering existing human-centered legal tools to assess AI liability. However, the analogy is inappropriate and misrepresents salient legal differences in how harms are committed by humans and AI systems. Thus, “as is often the …

Influence Level Prediction On Social Media Through Multi-Task And Sociolinguistic User Characteristics Modeling, Denys Katerenchuk

Dissertations, Theses, and Capstone Projects

Prediction of a user’s influence level on social networks has attracted a lot of attention as human interactions move online. Influential users have the ability to influence others’ behavior to achieve their own agenda. As a result, predicting users’ level of influence online can help to understand social networks, forecast trends, prevent misinformation, etc. The research on user influence in social networks has attracted much attention across multiple disciplines, from social sciences to mathematics, yet it is still not well understood. One of the difficulties is that the definition of influence is specific to a particular problem or a domain, …

The Interaction Of Different Primary Producers And Physical And Chemical Dynamics Of An Urban Shallow Lake, Majid Sahin

Dissertations, Theses, and Capstone Projects

An artificial urban shallow lake, Prospect Park Lake (PPL), is situated on a terminal moraine in Brooklyn New York, and supplied with municipal water treated with ortho-phosphates. The constant input of the phosphate nutrient is the primary source of eutrophication in the lake. The numerous pools along the water course houses various aquatic phototrophs, which influence the water quality and the state of the system, driving conditions into favoring the survival of their species. In the first half of the dissertation, the focus of the project is on analyzing how the different primary producers in different regions of PPL affect …

Finite Gaussian Neurons: Defending Against Adversarial Attacks By Making Neural Networks Say "I Don’T Know", Felix Grezes

Dissertations, Theses, and Capstone Projects

In this work, I introduce the Finite Gaussian Neuron (FGN), a novel neuron architecture for artificial neural networks aimed at protecting against adversarial attacks.
Since 2014, artificial neural networks have been known to be vulnerable to adversarial attacks, which can fool the network into producing wrong or nonsensical outputs by making humanly imperceptible alterations to inputs. While defenses against adversarial attacks have been proposed, they usually involve retraining a new neural network from scratch, a costly task.

My works aims to:
- easily convert existing models to Finite Gaussian Neuron architecture,
- while preserving the existing model's behavior on real …

Data-Centric Machine Learning For Speech And Audio, Ali Raza Syed

Dissertations, Theses, and Capstone Projects

There is growing recognition of the importance of data-centric methods for building machine learning systems. Data-centric methods assume a fixed model and iterate over the data to improve system performance. This is in contrast to traditional model-centric approaches, which assume a fixed dataset and iterate over models for the same ends. Data-centric machine learning is driven by the observation that, beyond the size of the training data, model performance depends on factors such as the quality of the annotations, and whether the data are representative of conditions in which models will be deployed. This is particularly of interest in the …

Two-Phase Matheuristic For The Vehicle Routing Problem With Reverse Cross-Docking, Aldy Gunawan, Audrey Tedja Widjaja, Pieter Vansteenwegen, Vincent F. Yu

Research Collection School Of Computing and Information Systems

Cross-dockingis a useful concept used by many companies to control the product flow. It enables the transshipment process of products from suppliers to customers. This research thus extends the benefit of cross-docking with reverse logistics, since return process management has become an important field in various businesses. The vehicle routing problem in a distribution network is considered to be an integrated model, namely the vehicle routing problem with reverse cross-docking (VRP-RCD). This study develops a mathematical model to minimize the costs of moving products in a four-level supply chain network that involves suppliers, cross-dock, customers, and outlets. A matheuristic based …

Learning To Solve Multiple-Tsp With Time Window And Rejections Via Deep Reinforcement Learning, Rongkai Zhang, Cong Zhang, Zhiguang Cao, Wen Song, Puay Siew Tan, Jie Zhang, Bihan Wen, Justin Dauwels

Research Collection School Of Computing and Information Systems

We propose a manager-worker framework (the implementation of our model is publically available at: https://github.com/zcaicaros/manager-worker-mtsptwr) based on deep reinforcement learning to tackle a hard yet nontrivial variant of Travelling Salesman Problem (TSP), i.e. multiple-vehicle TSP with time window and rejections (mTSPTWR), where customers who cannot be served before the deadline are subject to rejections. Particularly, in the proposed framework, a manager agent learns to divide mTSPTWR into sub-routing tasks by assigning customers to each vehicle via a Graph Isomorphism Network (GIN) based policy network. A worker agent learns to solve sub-routing tasks by minimizing the cost in terms of both …

Singapore Public Sector Ai Applications Emphasizing Public Engagement: Six Examples, Steven M. Miller

Research Collection School Of Computing and Information Systems

This article provides an overview of six examples of public sector AI applications in Singapore that illustrate different ways of enhancing engagement with the public. These applications demonstrate ways of enhancing engagement with the public by providing greater accessibility to government services (access anywhere, anytime) and speedier responses to public processes and feedback. Some applications make it substantially easier for members of the public to do things or make choices, while others reduce waiting time, either across an entire public infrastructure, or for an individual transaction. Some provide highly individualized coaching to guide a person through the process of doing …

Deep Learning-Based Text Recognition Of Agricultural Regulatory Document, Hua Leong Fwa, Farn Haur Chan

Research Collection School Of Computing and Information Systems

In this study, an OCR system based on deep learning techniques was deployed to digitize scanned agricultural regulatory documents comprising of certificates and labels. Recognition of the certificates and labels is challenging as they are scanned images of the hard copy form and the layout and size of the text as well as the languages vary between the various countries (due to diverse regulatory requirements). We evaluated and compared between various state-of-the-art deep learningbased text detection and recognition model as well as a packaged OCR library – Tesseract. We then adopted a two-stage approach comprising of text detection using Character …

Deep Learning For Coverage-Guided Fuzzing: How Far Are We?, Siqi Li, Xiaofei Xie, Yun Lin, Yuekang Li, Ruitao Feng, Xiaohong Li, Weimin Ge, Jin Song Dong

Research Collection School Of Computing and Information Systems

Fuzzing is a widely-used software vulnerability discovery technology, many of which are optimized using coverage-feedback. Recently, some techniques propose to train deep learning (DL) models to predict the branch coverage of an arbitrary input owing to its always-available gradients etc. as a guide. Those techniques have proved their success in improving coverage and discovering bugs under different experimental settings. However, DL models, usually as a magic black-box, are notoriously lack of explanation. Moreover, their performance can be sensitive to the collected runtime coverage information for training, indicating potentially unstable performance. In this work, we conduct a systematic empirical study on …

Contrastive Transformer-Based Multiple Instance Learning For Weakly Supervised Polyp Frame Detection, Tian Yu, Guansong Pang, Fengbei Liu, Yuyuan Liu, Chong Wang, Yuanhong Chen, Johan Verjans, Gustavo Carneiro

Research Collection School Of Computing and Information Systems

Current polyp detection methods from colonoscopy videos use exclusively normal (i.e., healthy) training images, which i) ignore the importance of temporal information in consecutive video frames, and ii) lack knowledge about the polyps. Consequently, they often have high detection errors, especially on challenging polyp cases (e.g., small, flat, or partially visible polyps). In this work, we formulate polyp detection as a weakly-supervised anomaly detection task that uses video-level labelled training data to detect frame-level polyps. In particular, we propose a novel convolutional transformer-based multiple instance learning method designed to identify abnormal frames (i.e., frames with polyps) from anomalous videos (i.e., …

A Carbon-Aware Planning Framework For Production Scheduling In Mining, Nurual Asyikeen Azhar, Aldy Gunawan, Shih-Fen Cheng, Erwin Leonardi

Research Collection School Of Computing and Information Systems

Managing the flow of excavated materials from a mine pit and the subsequent processing steps is the logistical challenge in mining. Mine planning needs to consider various geometric and resource constraints while maximizing the net present value (NPV) of profits over a long horizon. This mine planning problem has been modelled and solved as a precedence constrained production scheduling problem (PCPSP) using heuristics, due to its NP-hardness. However, the recent push for sustainable and carbon-aware mining practices calls for new planning approaches. In this paper, we propose an efficient temporally decomposed greedy Lagrangian relaxation (TDGLR) approach to maximize profits while …

Constrained Multiagent Reinforcement Learning For Large Agent Population, Jiajing Ling, Arambam James Singh, Duc Thien Nguyen, Akshat Kumar

Research Collection School Of Computing and Information Systems

Learning control policies for a large number of agents in a decentralized setting is challenging due to partial observability, uncertainty in the environment, and scalability challenges. While several scalable multiagent RL (MARL) methods have been proposed, relatively few approaches exist for large scale constrained MARL settings. To address this, we first formulate the constrained MARL problem in a collective multiagent setting where interactions among agents are governed by the aggregate count and types of agents, and do not depend on agents’ specific identities. Second, we show that standard Lagrangian relaxation methods, which are popular for single agent RL, do not …

Risk-Aware Procurement Optimization In A Global Technology Supply Chain, Jonathan Chase, Jingfeng Yang, Hoong Chuin Lau

Research Collection School Of Computing and Information Systems

Supply chain disruption, from ‘Black Swan’ events like the COVID-19 pandemic or the Russian invasion of Ukraine, to more ordinary issues such as labour disputes and adverse weather conditions, can result in delays, missed orders, and financial loss for companies that deliver products globally. Developing a risk-tolerant procurement strategy that anticipates the logistical problems incurred by disruption involves both accurate quantification of risk and cost-effective decision-making. We develop a supplier-focused risk evaluation metric that constrains a procurement optimization model for a global technology company. Our solution offers practical risk tolerance and cost-effectiveness, accounting for a range of constraints that realistically …

Truncated Matrix Power Iteration For Differentiable Dag Learning, Zhen Zhang, Ignavier Ng, Dong Gong, Yuhang Liu, Ehsan M. Abbasnejad, Mingming Gong, Kun Zhang, Javen Qinfeng Shi

Machine Learning Faculty Publications

Recovering underlying Directed Acyclic Graph structures (DAG) from observational data is highly challenging due to the combinatorial nature of the DAG-constrained optimization problem. Recently, DAG learning has been cast as a continuous optimization problem by characterizing the DAG constraint as a smooth equality one, generally based on polynomials over adjacency matrices. Existing methods place very small coefficients on high-order polynomial terms for stabilization, since they argue that large coefficients on the higher-order terms are harmful due to numeric exploding. On the contrary, we discover that large coefficients on higher-order terms are beneficial for DAG learning, when the spectral radiuses of …

Exploiting Higher-Order Derivatives In Convex Optimization Methods, Dmitry Kamzolov, Alexander Gasnikov, Pavel Dvurechensky, Artem Agafonov, Martin Takac

Machine Learning Faculty Publications

Exploiting higher-order derivatives in convex optimization is known at least since 1970’s. In each iteration higher-order (also called tensor) methods minimize a regularized Taylor expansion of the objective function, which leads to faster convergence rates if the corresponding higher-order derivative is Lipschitz-continuous. Recently a series of lower iteration complexity bounds for such methods were proved, and a gap between upper an lower complexity bounds was revealed. Moreover, it was shown that such methods can be implementable since the appropriately regularized Taylor expansion of a convex function is also convex and, thus, can be minimized in polynomial time. Only very recently …

Sr-Dcsk Cooperative Communication System With Code Index Modulation: A New Design For 6g New Radios, Yi Fang, Wang Chen, Pingping Chen, Yiwei Tao, Mohsen Guizani

Machine Learning Faculty Publications

This paper proposes a high-throughput short reference differential chaos shift keying cooperative communication system with the aid of code index modulation, referred to as CIM-SR-DCSK-CC system. In the proposed CIM-SR-DCSK-CC system, the source transmits information bits to both the relay and destination in the first time slot, while the relay not only forwards the source information bits but also sends new information bits to the destination in the second time slot. To be specific, the relay employs an N-order Walsh code to carry additional log2N information bits, which are superimposed onto the SR-DCSK signal carrying the decoded source information bits. …

Interpreting Song Lyrics With An Audio-Informed Pre-Trained Language Model, Yixiao Zhang, Junyan Jiang, Gus Xia, Simon Dixon

Machine Learning Faculty Publications

Lyric interpretations can help people understand songs and their lyrics quickly, and can also make it easier to manage, retrieve and discover songs efficiently from the growing mass of music archives. In this paper we propose BART-fusion, a novel model for generating lyric interpretations from lyrics and music audio that combines a large-scale pre-trained language model with an audio encoder. We employ a cross-modal attention module to incorporate the audio representation into the lyrics representation to help the pre-trained language model understand the song from an audio perspective, while preserving the language model’s original generative performance. We also release the …

Fdrl Approach For Association And Resource Allocation In Multi-Uav Air-To-Ground Iomt Network, Abegaz Mohammed, Aiman Erbad, Hayla Nahom, Abdullatif Albaseer, Mohammed Abdallah, Mohsen Guizani

Machine Learning Faculty Publications

In 6G networks, unmanned aerial vehicles (UAVs) can serve as aerial flying base stations (AFBS) with aerial mobile edge computing (AMEC) server capabilities. AFBS is an increasingly popular solution for delivering time-sensitive applications, extending network coverage, and assisting ground base stations in the healthcare systems for remote areas with limited infrastructure. Furthermore, the UAVs are deployed in the healthcare system to support the Internet of medical things (IoMT) devices in data collection, medical equipment distribution, and providing smart services. However, ensuring the privacy and security of patients’ data with the limited UAV resources is a major challenge. In this paper, …

Reconfigurable Intelligent Surfaces And Capacity Optimization: A Large System Analysis, Aris L. Moustakas, George C. Alexandropoulos, Mérouane Debbah

Machine Learning Faculty Publications

Reconfigurable Intelligent Surfaces (RISs), comprising large numbers of low-cost and almost passive metamaterials with tunable reflection properties, have been recently proposed as an enabling technology for programmable wireless propagation environments. In this paper, we present asymptotic closed-form expressions for the mean and variance of the mutual information metric for a multi-antenna transmitter-receiver pair in the presence of multiple RISs, using methods from statistical physics. While nominally valid in the large system limit, we show that the derived Gaussian approximation for the mutual information can be quite accurate, even for modest-sized antenna arrays and metasurfaces. The above results are particularly useful …

Transformnet: Self-Supervised Representation Learning Through Predicting Geometric Transformations, Muhammad Ali, Sayed Hashim

Computer Vision Faculty Publications

Deep neural networks need a big amount of training data, while in the real world there is a scarcity of data available for training purposes. To resolve this issue unsupervised methods are used for training with limited data. In this report, we describe the unsupervised semantic feature learning approach for recognition of the geometric transformation applied to the input data. The basic concept of our approach is that if someone is unaware of the objects in the images, he/she would not be able to quantitatively predict the geometric transformation that was applied to them. This self supervised scheme is based …

An Analysis Of Android Malware Detection Using Tree Learning Techniques, Kyler D. Dickey

Student Theses and Dissertations

Android malware is a growing threat, coinciding with the increasing adoption of the Android platform. Malware detection methods used to maintain user privacy and system integrity are increasingly becoming the subject of research. Many new methods studied employ learning algorithms to detect malicious programs. This study investigates the use of byte and opcode frequency features as inputs for tree-based machine learning methods. The algorithm is optimized to reduce overfitting given input hyperparameter combinations and is tuned using cross-validation procedures. Lastly, the study deliberates on possible avenues for future research to gather more concrete evidence for the efficacy and cost-effectiveness of …