Artificial Intelligence and Robotics Commons^™

Visualizing Routes With Ai-Discovered Street-View Patterns, Tsung Heng Wu, Md Amiruzzaman, Ye Zhao, Deepshikha Bhati, Jing Yang

Computer Science Faculty Publications

Street-level visual appearances play an important role in studying social systems, such as understanding the built environment, driving routes, and associated social and economic factors. It has not been integrated into a typical geographical visualization interface (e.g., map services) for planning driving routes. In this article, we study this new visualization task with several new contributions. First, we experiment with a set of AI techniques and propose a solution of using semantic latent vectors for quantifying visual appearance features. Second, we calculate image similarities among a large set of street-view images and then discover spatial imagery patterns. Third, we integrate …

Dilf: Differentiable Rendering-Based Multi-View Image-Language Fusion For Zero-Shot 3d Shape Understanding, Xin Ning, Zaiyang Yu, Lusi Li, Weijun Li, Prayag Tiwari

Computer Science Faculty Publications

Zero-shot 3D shape understanding aims to recognize “unseen” 3D categories that are not present in training data. Recently, Contrastive Language–Image Pre-training (CLIP) has shown promising open-world performance in zero-shot 3D shape understanding tasks by information fusion among language and 3D modality. It first renders 3D objects into multiple 2D image views and then learns to understand the semantic relationships between the textual descriptions and images, enabling the model to generalize to new and unseen categories. However, existing studies in zero-shot 3D shape understanding rely on predefined rendering parameters, resulting in repetitive, redundant, and low-quality views. This limitation hinders the model’s …

A Survey On Few-Shot Class-Incremental Learning, Songsong Tian, Lusi Li, Weijun Li, Hang Ran, Xin Ning, Prayag Tiwari

Computer Science Faculty Publications

Large deep learning models are impressive, but they struggle when real-time data is not available. Few-shot class-incremental learning (FSCIL) poses a significant challenge for deep neural networks to learn new tasks from just a few labeled samples without forgetting the previously learned ones. This setup can easily leads to catastrophic forgetting and overfitting problems, severely affecting model performance. Studying FSCIL helps overcome deep learning model limitations on data volume and acquisition time, while improving practicality and adaptability of machine learning models. This paper provides a comprehensive survey on FSCIL. Unlike previous surveys, we aim to synthesize few-shot learning and incremental …

A Chinese Power Text Classification Algorithm Based On Deep Active Learning, Song Deng, Qianliang Li, Renjie Dai, Siming Wei, Di Wu, Yi He, Xindong Wu

Computer Science Faculty Publications

The construction of knowledge graph is beneficial for grid production, electrical safety protection, fault diagnosis and traceability in an observable and controllable way. Highly-precision text classification algorithm is crucial to build a professional knowledge graph in power system. Unfortunately, there are a large number of poorly described and specialized texts in the power business system, and the amount of data containing valid labels in these texts is low. This will bring great challenges to improve the precision of text classification models. To offset the gap, we propose a classification algorithm for Chinese text in the power system based on deep …

Learning Optimal Inter-Class Margin Adaptively For Few-Shot Class-Incremental Learning Via Neural Collapse-Based Meta-Learning, Hang Ran, Weijun Li, Lusi Li, Songsong Tian, Xin Ning, Prayag Tiwari

Computer Science Faculty Publications

Few-Shot Class-Incremental Learning (FSCIL) aims to learn new classes incrementally with a limited number of samples per class. It faces issues of forgetting previously learned classes and overfitting on few-shot classes. An efficient strategy is to learn features that are discriminative in both base and incremental sessions. Current methods improve discriminability by manually designing inter-class margins based on empirical observations, which can be suboptimal. The emerging Neural Collapse (NC) theory provides a theoretically optimal inter-class margin for classification, serving as a basis for adaptively computing the margin. Yet, it is designed for closed, balanced data, not for sequential or few-shot …

Identifying Patterns For Neurological Disabilities By Integrating Discrete Wavelet Transform And Visualization, Soo Yeon Ji, Sampath Jayarathna, Anne M. Perrotti, Katrina Kardiasmenos, Dong Hyun Jeong

Computer Science Faculty Publications

Neurological disabilities cause diverse health and mental challenges, impacting quality of life and imposing financial burdens on both the individuals diagnosed with these conditions and their caregivers. Abnormal brain activity, stemming from malfunctions in the human nervous system, characterizes neurological disorders. Therefore, the early identification of these abnormalities is crucial for devising suitable treatments and interventions aimed at promoting and sustaining quality of life. Electroencephalogram (EEG), a non-invasive method for monitoring brain activity, is frequently employed to detect abnormal brain activity in neurological and mental disorders. This study introduces an approach that extends the understanding and identification of neurological disabilities …

Robots Still Outnumber Humans In Web Archives In 2019, But Less Than In 2015 And 2012, Himarsha R. Jayanetti, Kritika Garg, Sawood Alam, Michael L. Nelson, Michele C. Weigle

Computer Science Faculty Publications

The significance of the web and the crucial role of web archives in its preservation highlight the necessity of understanding how users, both human and robot, access web archive content, and how best to satisfy this disparate needs of both types of users. To identify robots and humans in web archives and analyze their respective access patterns, we used the Internet Archive’s (IA) Wayback Machine access logs from 2012, 2015, and 2019, as well as Arquivo.pt’s (Portuguese Web Archive) access logs from 2019. We identified user sessions in the access logs and classified those sessions as human or robot based …

Triphlapan: Predicting Hla Molecules Binding Peptides Based On Triple Coding Matrix And Transfer Learning, Meng Wang, Chuqi Lei, Jianxin Wang, Yaohang Li, Min Li

Computer Science Faculty Publications

Human leukocyte antigen (HLA) recognizes foreign threats and triggers immune responses by presenting peptides to T cells. Computationally modeling the binding patterns between peptide and HLA is very important for the development of tumor vaccines. However, it is still a big challenge to accurately predict HLA molecules binding peptides. In this paper, we develop a new model TripHLApan for predicting HLA molecules binding peptides by integrating triple coding matrix, BiGRU + Attention models, and transfer learning strategy. We have found the main interaction site regions between HLA molecules and peptides, as well as the correlation between HLA encoding and binding …

An Ai-Based Framework For Translating American Sign Language To English And Vice Versa, Vijayendra D. Avina, Md Amiruzzaman, Stefanie Amiruzzaman, Linh B. Ngo, M. Ali Akber Dewan

Computer Science Faculty Publications

Abstract: In this paper, we propose a framework to convert American Sign Language (ASL) to English and English to ASL. Within this framework, we use a deep learning model along with the rolling average prediction that captures image frames from videos and classifies the signs from the image frames. The classified frames are then used to construct ASL words and sentences to support people with hearing impairments. We also use the same deep learning model to capture signs from the people with deaf symptoms and convert them into ASL words and English sentences. Based on this framework, we developed a …

Dfhic: A Dilated Full Convolution Model To Enhance The Resolution Of Hi-C Data, Bin Wang, Kun Liu, Yaohang Li, Jianxin Wang

Computer Science Faculty Publications

Motivation: Hi-C technology has been the most widely used chromosome conformation capture(3C) experiment that measures the frequency of all paired interactions in the entire genome, which is a powerful tool for studying the 3D structure of the genome. The fineness of the constructed genome structure depends on the resolution of Hi-C data. However, due to the fact that high-resolution Hi-C data require deep sequencing and thus high experimental cost, most available Hi-C data are in low-resolution. Hence, it is essential to enhance the quality of Hi-C data by developing the effective computational methods.

Results: In this work, we propose …

Mitigating Anomalous Electricity Consumption In Smart Cities Using An Ai-Based Stacked-Generalization Technique, Arshid Ali, Laiq Khan, Nadeem Javaid, Safdar Hussain Bouk, Abdulaziz Aldegheishem, Nabil Alrahjeh

Computer Science Faculty Publications

Energy management and efficient asset utilization play an important role in the economic development of a country. The electricity produced at the power station faces two types of losses from the generation point to the end user. These losses are technical losses (TL) and non-technical losses (NTL). TLs occurs due to the use of inefficient equipment. While NTLs occur due to the anomalous consumption of electricity by the customers, which happens in many ways; energy theft being one of them. Energy theft majorly happens to cut down on the electricity bills. These losses in the smart grid (SG) are the …

Msdrp: A Deep Learning Model Based On Multisource Data For Predicting Drug Response, Haochen Zhao, Xiaoyu Zhang, Qichang Zhao, Yaohang Li, Jianxin Wang

Computer Science Faculty Publications

Motivation: Cancer heterogeneity drastically affects cancer therapeutic outcomes. Predicting drug response in vitro is expected to help formulate personalized therapy regimens. In recent years, several computational models based on machine learning and deep learning have been proposed to predict drug response in vitro. However, most of these methods capture drug features based on a single drug description (e.g. drug structure), without considering the relationships between drugs and biological entities (e.g. target, diseases, and side effects). Moreover, most of these methods collect features separately for drugs and cell lines but fail to consider the pairwise interactions between drugs and cell …

Toward A Generative Modeling Analysis Of Clas Exclusive 2𝜋 Photoproduction, T. Alghamdi, Y. Alanazi, M. Battaglieri, Ł. Bibrzycki, A. V. Golda, A. N. Hiller Blin, E. L. Isupov, Y. Li, L. Marsicano, W. Melnitchouk, V. I. Mokeev, G. Montaña, A. Pilloni, N. Sato, A. P. Szczepaniak, T. Vittorini

Computer Science Faculty Publications

AI-supported algorithms, particularly generative models, have been successfully used in a variety of different contexts. This work employs a generative modeling approach to unfold detector effects specifically tailored for exclusive reactions that involve multiparticle final states. Our study demonstrates the preservation of correlations between kinematic variables in a multidimensional phase space. We perform a full closure test on two-pion photoproduction pseudodata generated with a realistic model in the kinematics of the Jefferson Lab CLAS g11 experiment. The overlap of different reaction mechanisms leading to the same final state associated with the CLAS detector’s nontrivial effects represents an ideal test case …

A Hybrid Deep Learning Approach For Crude Oil Price Prediction, Hind Aldabagh, Xianrong Zheng, Ravi Mukkamala

Computer Science Faculty Publications

Crude oil is one of the world’s most important commodities. Its price can affect the global economy, as well as the economies of importing and exporting countries. As a result, forecasting the price of crude oil is essential for investors. However, crude oil price tends to fluctuate considerably during significant world events, such as the COVID-19 pandemic and geopolitical conflicts. In this paper, we propose a deep learning model for forecasting the crude oil price of one-step and multi-step ahead. The model extracts important features that impact crude oil prices and uses them to predict future prices. The prediction model …

A Structure-Aware Generative Adversarial Network For Bilingual Lexicon Induction, Bocheng Han, Qian Tao, Lusi Li, Zhihao Xiong

Computer Science Faculty Publications

Bilingual lexicon induction (BLI) is the task of inducing word translations with a learned mapping function that aligns monolingual word embedding spaces in two different languages. However, most previous methods treat word embeddings as isolated entities and fail to jointly consider both the intra-space and inter-space topological relations between words. This limitation makes it challenging to align words from embedding spaces with distinct topological structures, especially when the assumption of isomorphism may not hold. To this end, we propose a novel approach called the Structure-Aware Generative Adversarial Network (SA-GAN) model to explicitly capture multiple topological structure information to achieve accurate …

Charged Track Reconstruction With Artificial Intelligence For Clas12, Gagik Gavalian, Polykarpos Thomadakis, Angelos Angelopoulos, Nikos Chrisochoides

Computer Science Faculty Publications

In this paper, we present the results of charged particle track reconstruction in CLAS12 using artificial intelligence. In our approach, we use neural networks working together to identify tracks based on the raw signals in the Drift Chambers. A Convolutional Auto-Encoder is used to de-noise raw data by removing the hits that do not satisfy the patterns for tracks, and second Multi-Layer Perceptron is used to identify tracks from combinations of clusters in the drift chambers. Our method increases the tracking efficiency by 50% for multi-particle final states already conducted experiments. The de-noising results indicate that future experiments can run …

An Approach To Developing Benchmark Datasets For Protein Secondary Structure Segmentation From Cryo-Em Density Maps, Thu Nguyen, Yongcheng Mu, Jiangwen Sun, Jing He

Computer Science Faculty Publications

More and more deep learning approaches have been proposed to segment secondary structures from cryo-electron density maps at medium resolution range (5--10Å). Although the deep learning approaches show great potential, only a few small experimental data sets have been used to test the approaches. There is limited understanding about potential factors, in data, that affect the performance of segmentation. We propose an approach to generate data sets with desired specifications in three potential factors - the protein sequence identity, structural contents, and data quality. The approach was implemented and has generated a test set and various training sets to study …

Identifying The Serious Clinical Outcomes Of Adverse Reactions To Drugs By A Multi-Task Deep Learning Framework, Haochen Zhao, Peng Ni, Qichang Zhao, Xiao Liang, Di Ai, Shannon Erhardt, Jun Wang, Yaohang Li, Jiianxin Wang

Computer Science Faculty Publications

Adverse Drug Reactions (ADRs) have a direct impact on human health. As continuous pharmacovigilance and drug monitoring prove to be costly and time-consuming, computational methods have emerged as promising alternatives. However, most existing computational methods primarily focus on predicting whether or not the drug is associated with an adverse reaction and do not consider the core issue of drug benefit-risk assessment-whether the treatment outcome is serious when adverse drug reactions occur. To this end, we categorize serious clinical outcomes caused by adverse reactions to drugs into seven distinct classes and present a deep learning framework, so-called GCAP, for predicting the …

Claimdistiller: Scientific Claim Extraction With Supervised Contrastive Learning, Xin Wei, Md Reshad Ul Hoque, Jian Wu, Jiang Li

Computer Science Faculty Publications

The growth of scientific papers in the past decades calls for effective claim extraction tools to automatically and accurately locate key claims from unstructured text. Such claims will benefit content-wise aggregated exploration of scientific knowledge beyond the metadata level. One challenge of building such a model is how to effectively use limited labeled training data. In this paper, we compared transfer learning and contrastive learning frameworks in terms of performance, time and training data size. We found contrastive learning has better performance at a lower cost of data across all models. Our contrastive-learning-based model ClaimDistiller has the highest performance, boosting …

An Ai-Based Framework For Studying Visual Diversity Of Urban Neighborhoods And Its Relationship With Socio-Demographic Variables, Md Amiruzzaman, Ye Zhao, Stefanie Amiruzzaman, Aryn C. Karpinski, Tsung Heng Wu

Computer Science Faculty Publications

This study presents a framework to study quantitatively geographical visual diversities of urban neighborhood from a large collection of street-view images using an Artificial Intelligence (AI)-based image segmentation technique. A variety of diversity indices are computed from the extracted visual semantics. They are utilized to discover the relationships between urban visual appearance and socio-demographic variables. This study also validates the reliability of the method with human evaluators. The methodology and results obtained from this study can potentially be used to study urban features, locate houses, establish services, and better operate municipalities.

Can Lethal Autonomous Weapons Be Just?, Noreen L. Herzfeld, Robert H. Latiff

Computer Science Faculty Publications

In 2018 the United States Department of Defense (DoD) created a new Joint Artificial Intelligence Center to study the adoption of AI by the military. Their strategy, outlined in a document entitled, “Harnessing AI to Advance Our Security and Prosperity,” proposes to accelerate the adoption of AI in the military by fostering a culture of experimentation and calculated risk taking, noting that AI will change the character of the future battlefield and, even more, the pace of battle. Is there any way to ensure that this future battlefield will be just? Can the age-old precepts of just warfare help guide …

Introducing A Real-Time Advanced Eye Movements Analysis Pipeline, Gavindya Jayawardana

Computer Science Faculty Publications

Real-Time Advanced Eye Movements Analysis Pipeline (RAEMAP) is an advanced pipeline to analyze traditional positional gaze measurements as well as advanced eye gaze measurements. The proposed implementation of RAEMAP includes real-time analysis of fixations, saccades, gaze transition entropy, and low/high index of pupillary activity. RAEMAP will also provide visualizations of fixations, fixations on AOIs, heatmaps, and dynamic AOI generation in real-time. This paper outlines the proposed architecture of RAEMAP.

Visual Descriptor Extraction From Patent Figure Captions: A Case Study Of Data Efficiency Between Bilstm And Transformer, Xin Wei, Jian Wu, Kehinde Ajayi, Diane Oyen

Computer Science Faculty Publications

Technical drawings used for illustrating designs are ubiquitous in patent documents, especially design patents. Different from natural images, these drawings are usually made using black strokes with little color information, making it challenging for models trained on natural images to recognize objects. To facilitate indexing and searching, we propose an effective and efficient visual descriptor model that extracts object names and aspects from patent captions to annotate benchmark patent figure datasets. We compared two state-of-the-art named entity recognition (NER) models and found that with a limited number of annotated samples, the BiLSTM-CRF model outperforms the Transformer model by a significant …

Customer Gaze Estimation In Retail Using Deep Learning, Shashimal Senarath, Primesh Pathirana, Dulani Meedeniya, Sampath Jayarathna

Computer Science Faculty Publications

At present, intelligent computing applications are widely used in different domains, including retail stores. The analysis of customer behaviour has become crucial for the benefit of both customers and retailers. In this regard, the concept of remote gaze estimation using deep learning has shown promising results in analyzing customer behaviour in retail due to its scalability, robustness, low cost, and uninterrupted nature. This study presents a three-stage, three-attention-based deep convolutional neural network for remote gaze estimation in retail using image data. In the first stage, we design a mechanism to estimate the 3D gaze of the subject using image data …

Machine Learning-Based Event Generator For Electron-Proton Scattering, Y. Alanazi, P. Ambrozewicz, M. Battaglieri, A.N. Hiller Blin, M. P. Kuchera, Y. Li, T. Liu, R. E. Mcclellan, W. Melnitchouk, E. Pritchard, M. Robertson, N. Sato, R. Strauss, L. Velasco

Computer Science Faculty Publications

We present a new machine learning-based Monte Carlo event generator using generative adversarial networks (GANs) that can be trained with calibrated detector simulations to construct a vertex-level event generator free of theoretical assumptions about femtometer scale physics. Our framework includes a GAN-based detector folding as a fast-surrogate model that mimics detector simulators. The framework is tested and validated on simulated inclusive deep-inelastic scattering data along with existing parametrizations for detector simulation, with uncertainty quantification based on a statistical bootstrapping technique. Our results provide for the first time a realistic proof of concept to mitigate theory bias in inferring vertex-level event …

A Synthetic Prediction Market For Estimating Confidence In Published Work, Sarah Rajtmajer, Christopher Griffin, Jian Wu, Robert Fraleigh, Laxmann Balaji, Anna Squicciarini, Anthony Kwasnica, David Pennock, Michael Mclaughlin, Timothy Fritton, Nishanth Nakshatri, Arjun Menon, Sai Ajay Modukuri, Rajal Nivargi, Xin Wei, Lee Giles

Computer Science Faculty Publications

[First paragraph] Concerns about the replicability, robustness and reproducibility of findings in scientific literature have gained widespread attention over the last decade in the social sciences and beyond. This attention has been catalyzed by and has likewise motivated a number of large-scale replication projects which have reported successful replication rates between 36% and 78%. Given the challenges and resources required to run high-powered replication studies, researchers have sought other approaches to assess confidence in published claims. Initial evidence has supported the promise of prediction markets in this context. However, they require the coordinated, sustained effort of collections of human experts …

Online Deep Learning From Doubly-Streaming Data, Heng Lian, John S. Atwood, Bo-Jian Hou, Jian Wu, Yi He

Computer Science Faculty Publications

This paper investigates a new online learning problem with doubly-streaming data, where the data streams are described by feature spaces that constantly evolve, with new features emerging and old features fading away. A plausible idea to deal with such data streams is to establish a relationship between the old and new feature spaces, so that an online learner can leverage the knowledge learned from the old features to better the learning performance on the new features. Unfortunately, this idea does not scale up to high-dimensional multimedia data with complex feature interplay, which suffers a tradeoff between onlineness, which biases shallow …

Theory Entity Extraction For Social And Behavioral Sciences Papers Using Distant Supervision, Xin Wei, Lamia Salsabil, Jian Wu

Computer Science Faculty Publications

Theories and models, which are common in scientific papers in almost all domains, usually provide the foundations of theoretical analysis and experiments. Understanding the use of theories and models can shed light on the credibility and reproducibility of research works. Compared with metadata, such as title, author, keywords, etc., theory extraction in scientific literature is rarely explored, especially for social and behavioral science (SBS) domains. One challenge of applying supervised learning methods is the lack of a large number of labeled samples for training. In this paper, we propose an automated framework based on distant supervision that leverages entity mentions …

Multi-User Eye-Tracking, Bhanuka Mahanama

Computer Science Faculty Publications

The human gaze characteristics provide informative cues on human behavior during various activities. Using traditional eye trackers, assessing gaze characteristics in the wild requires a dedicated device per participant and therefore is not feasible for large-scale experiments. In this study, we propose a commodity hardware-based multi-user eye-tracking system. We leverage the recent advancements in Deep Neural Networks and large-scale datasets for implementing our system. Our preliminary studies provide promising results for multi-user eye-tracking on commodity hardware, providing a cost-effective solution for large-scale studies.

Camouflaged Poisoning Attack On Graph Neural Networks, Chao Jiang, Yi He, Richard Chapman, Hongyi Wu

Computer Science Faculty Publications

Graph neural networks (GNNs) have enabled the automation of many web applications that entail node classification on graphs, such as scam detection in social media and event prediction in service networks. Nevertheless, recent studies revealed that the GNNs are vulnerable to adversarial attacks, where feeding GNNs with poisoned data at training time can lead them to yield catastrophically devastative test accuracy. This finding heats up the frontier of attacks and defenses against GNNs. However, the prior studies mainly posit that the adversaries can enjoy free access to manipulate the original graph, while obtaining such access could be too costly in …