Artificial Intelligence and Robotics Commons^™

Visual Descriptor Extraction From Patent Figure Captions: A Case Study Of Data Efficiency Between Bilstm And Transformer, Xin Wei, Jian Wu, Kehinde Ajayi, Diane Oyen

Computer Science Faculty Publications

Technical drawings used for illustrating designs are ubiquitous in patent documents, especially design patents. Different from natural images, these drawings are usually made using black strokes with little color information, making it challenging for models trained on natural images to recognize objects. To facilitate indexing and searching, we propose an effective and efficient visual descriptor model that extracts object names and aspects from patent captions to annotate benchmark patent figure datasets. We compared two state-of-the-art named entity recognition (NER) models and found that with a limited number of annotated samples, the BiLSTM-CRF model outperforms the Transformer model by a significant …

Camouflaged Poisoning Attack On Graph Neural Networks, Chao Jiang, Yi He, Richard Chapman, Hongyi Wu

Computer Science Faculty Publications

Graph neural networks (GNNs) have enabled the automation of many web applications that entail node classification on graphs, such as scam detection in social media and event prediction in service networks. Nevertheless, recent studies revealed that the GNNs are vulnerable to adversarial attacks, where feeding GNNs with poisoned data at training time can lead them to yield catastrophically devastative test accuracy. This finding heats up the frontier of attacks and defenses against GNNs. However, the prior studies mainly posit that the adversaries can enjoy free access to manipulate the original graph, while obtaining such access could be too costly in …

Machine Learning-Based Event Generator For Electron-Proton Scattering, Y. Alanazi, P. Ambrozewicz, M. Battaglieri, A.N. Hiller Blin, M. P. Kuchera, Y. Li, T. Liu, R. E. Mcclellan, W. Melnitchouk, E. Pritchard, M. Robertson, N. Sato, R. Strauss, L. Velasco

Computer Science Faculty Publications

We present a new machine learning-based Monte Carlo event generator using generative adversarial networks (GANs) that can be trained with calibrated detector simulations to construct a vertex-level event generator free of theoretical assumptions about femtometer scale physics. Our framework includes a GAN-based detector folding as a fast-surrogate model that mimics detector simulators. The framework is tested and validated on simulated inclusive deep-inelastic scattering data along with existing parametrizations for detector simulation, with uncertainty quantification based on a statistical bootstrapping technique. Our results provide for the first time a realistic proof of concept to mitigate theory bias in inferring vertex-level event …

Online Deep Learning From Doubly-Streaming Data, Heng Lian, John S. Atwood, Bo-Jian Hou, Jian Wu, Yi He

Computer Science Faculty Publications

This paper investigates a new online learning problem with doubly-streaming data, where the data streams are described by feature spaces that constantly evolve, with new features emerging and old features fading away. A plausible idea to deal with such data streams is to establish a relationship between the old and new feature spaces, so that an online learner can leverage the knowledge learned from the old features to better the learning performance on the new features. Unfortunately, this idea does not scale up to high-dimensional multimedia data with complex feature interplay, which suffers a tradeoff between onlineness, which biases shallow …

Theory Entity Extraction For Social And Behavioral Sciences Papers Using Distant Supervision, Xin Wei, Lamia Salsabil, Jian Wu

Computer Science Faculty Publications

Theories and models, which are common in scientific papers in almost all domains, usually provide the foundations of theoretical analysis and experiments. Understanding the use of theories and models can shed light on the credibility and reproducibility of research works. Compared with metadata, such as title, author, keywords, etc., theory extraction in scientific literature is rarely explored, especially for social and behavioral science (SBS) domains. One challenge of applying supervised learning methods is the lack of a large number of labeled samples for training. In this paper, we propose an automated framework based on distant supervision that leverages entity mentions …

A Synthetic Prediction Market For Estimating Confidence In Published Work, Sarah Rajtmajer, Christopher Griffin, Jian Wu, Robert Fraleigh, Laxmann Balaji, Anna Squicciarini, Anthony Kwasnica, David Pennock, Michael Mclaughlin, Timothy Fritton, Nishanth Nakshatri, Arjun Menon, Sai Ajay Modukuri, Rajal Nivargi, Xin Wei, Lee Giles

Computer Science Faculty Publications

[First paragraph] Concerns about the replicability, robustness and reproducibility of findings in scientific literature have gained widespread attention over the last decade in the social sciences and beyond. This attention has been catalyzed by and has likewise motivated a number of large-scale replication projects which have reported successful replication rates between 36% and 78%. Given the challenges and resources required to run high-powered replication studies, researchers have sought other approaches to assess confidence in published claims. Initial evidence has supported the promise of prediction markets in this context. However, they require the coordinated, sustained effort of collections of human experts …

Morphologic Clustering Of Earcanals Using Deep Learning Algorithm To Design Artificial Ears Dedicated To Earplugs Attenuation Measurement, Bastien Poissenot-Arrigoni, Chun Hong Law, Djamal Berbiche, Franck Sgard, Olivier Doutres

Études primaires

Designing earplugs adapted for the widest number of earcanals requires acoustical test fixtures (ATFs) geometrically representative of the population. Most existing ATFs are equipped with unique sized straight cylindrical earcanals, considered representative of average human morphology, and are therefore unable to assess how earplugs can fit different earcanal morphologies. In this study, a methodology to cluster earcanals as a function of their morphologies with the objective of designing artificial ears dedicated to sound attenuation measurement is developed and applied to a sample of Canadian workers’ earcanals. The earcanal morphologic indicators that correlate with the attenuations of six models of commercial …

An Autoencoder-Based Deep Learning Method For Genotype Imputation, Meng Song, Jonathan Greenbaum, Joseph Luttrell, Weihua Zhou, Chong Wu, Zhe Luo, Chuan Qiu, Lan Juan Zhao, Kuan-Jui Su, Qing Tian, Hui Shen, Huixiao Hong, Ping Gong, Xinghua Shi, Hong-Wen Deng, Chaoyang Zhang

Faculty, Staff and Student Publications

Genotype imputation has a wide range of applications in genome-wide association study (GWAS), including increasing the statistical power of association tests, discovering trait-associated loci in meta-analyses, and prioritizing causal variants with fine-mapping. In recent years, deep learning (DL) based methods, such as sparse convolutional denoising autoencoder (SCDA), have been developed for genotype imputation. However, it remains a challenging task to optimize the learning process in DL-based methods to achieve high imputation accuracy. To address this challenge, we have developed a convolutional autoencoder (AE) model for genotype imputation and implemented a customized training loop by modifying the training process with a …

Highlights Generation For Tennis Matches Using Computer Vision, Natural Language Processing And Audio Analysis, Alon Liberman

Senior Independent Study Theses

This project uses computer vision, natural language processing and audio analysis to automatize the highlights generation task for tennis matches. Computer vision techniques such as camera shot detection, hough transform and neural networks are used to extract the time intervals of the points. To detect the best points, three approaches are used. Point length suggests which points correspond to rallies and aces. The audio waves are analyzed to search for the highest audio peaks, which indicate the moments where the crowd cheers the most. Sentiment analysis, a natural language processing technique, is used to look for points where the commentators …

Measuring Vegetation Density In Marsh Grass Photographs Using Convolutional Neural Networks, Lucas W. Welch

UNF Graduate Theses and Dissertations

A marsh is a wetland dominated by various species of grasses rather than trees. It supports an important ecosystem, providing habitats for many kinds of animals and posing a crucial impact on coastal climates. Marshes are rapidly changing, and it is vital for scientists to track these changes to understand the health of them. To do so, biologists perform vegetation monitoring to estimate the coverage of vegetation in an area of marsh. This task often calls for extensive human labor carefully examining pixels in photos of marsh sites to calculate vegetation density for a variety of marsh grass species. This …

Advances And Applications In High-Dimensional Heuristic Optimization, Samuel Alexander Vanfossan

Doctoral Dissertations

“Applicable to most real-world decision scenarios, multiobjective optimization is an area of multicriteria decision-making that seeks to simultaneously optimize two or more conflicting objectives. In contrast to single-objective scenarios, nontrivial multiobjective optimization problems are characterized by a set of Pareto optimal solutions wherein no solution unanimously optimizes all objectives. Evolutionary algorithms have emerged as a standard approach to determine a set of these Pareto optimal solutions, from which a decision-maker can select a vetted alternative. While easy to implement and having demonstrated great efficacy, these evolutionary approaches have been criticized for their runtime complexity when dealing with many alternatives or …

Part I - Ai And Data As Medical Devices, W. Nicholson Price Ii

Other Publications

It may seem counterintuitive to open a book on medical devices with chapters on software and data, but these are the frontiers of new medical device regulation and law. Physical devices are still crucial to medicine, but they – and medical practice as a whole – are embedded in and permeated by networks of software and caches of data. Those software systems are often mindbogglingly complex and largely inscrutable, involving artificial intelligence and machine learning. Ensuring that such software works effectively and safely remains a substantial challenge for regulators and policymakers. Each of the three chapters in this part examines …

Secure Multi-Robot Adaptive Information Sampling With Continuous, Periodic And Opportunistic Connectivity, Tamim Khatib

UNF Graduate Theses and Dissertations

Multi-robot teams are an increasingly popular approach for information gathering in large geographic areas, with applications in precision agriculture, natural disaster aftermath surveying, and pollution tracking. In a coordinated multi-robot information sampling scenario, robots share their collected information amongst one another to form better predictions. These robot teams are often assembled from untrusted devices, making the verification of the integrity of the collected samples an important challenge. Furthermore, such robots often operate under conditions of continuous, periodic, or opportunistic connectivity and are limited in their energy budget and computational power. In this thesis, we study how to secure the information …

Towards An Active Foveated Approach To Computer Vision, Dario Dematties, Silvio Rizzi, George K. Thiruvathukal, Alejandro Javier Wainselboim

Computer Science: Faculty Publications and Other Works

In this paper, a series of experimental methods are presented explaining a new approach towards active foveated Computer Vision (CV). This is a collaborative effort between researchers at CONICET Mendoza Technological Scientific Center from Argentina, Argonne National Laboratory (ANL), and Loyola University Chicago from the US. The aim is to advance new CV approaches more in line with those found in biological agents in order to bring novel solutions to the main problems faced by current CV applications. Basically this work enhance Self-supervised (SS) learning, incorporating foveated vision plus saccadic behavior in order to improve training and computational efficiency without …

Exploring Algorithmic Literacy For College Students: An Educator’S Roadmap, Susan Gardner Archambault

LMU Theses and Dissertations

Research shows that college students are largely unaware of the impact of algorithms on their everyday lives. Also, most university students are not being taught about algorithms as part of the regular curriculum. This exploratory, qualitative study aimed to explore subject-matter experts’ insights and perceptions of the knowledge components, coping behaviors, and pedagogical considerations to aid faculty in teaching algorithmic literacy to college students. Eleven individual, semi-structured interviews and one focus group were conducted with scholars and teachers of critical algorithm studies and related fields. Findings suggested three sets of knowledge components that would contribute to students’ algorithmic literacy: general …

Application Of Artificial Intelligence To Plasma Metabolomics Profiles To Predict Response To Neoadjuvant Chemotherapy In Triple-Negative Breast Cancer, Ehsan Irajizad, Ranran Wu, Jody Vykoukal, Eunice Murage, Rachelle Spencer, Jennifer B Dennison, Stacy Moulder, Elizabeth Ravenberg, Bora Lim, Jennifer Litton, Debu Tripathym, Vicente Valero, Senthil Damodaran, Gaiane M Rauch, Beatriz Adrada, Rosalind Candelaria, Jason B White, Abenaa Brewster, Banu Arun, James P Long, Kim Anh Do, Sam Hanash, Johannes F Fahrmann

Faculty, Staff and Student Publications

There is a need to identify biomarkers predictive of response to neoadjuvant chemotherapy (NACT) in triple-negative breast cancer (TNBC). We previously obtained evidence that a polyamine signature in the blood is associated with TNBC development and progression. In this study, we evaluated whether plasma polyamines and other metabolites may identify TNBC patients who are less likely to respond to NACT. Pre-treatment plasma levels of acetylated polyamines were elevated in TNBC patients that had moderate to extensive tumor burden (RCB-II/III) following NACT compared to those that achieved a complete pathological response (pCR/RCB-0) or had minimal residual disease (RCB-I). We further applied …

Temporal Disambiguation Of Relative Temporal Expressions In Clinical Texts Using Temporally Fine-Tuned Contextual Word Embeddings., Amy L. Olex

Theses and Dissertations

Temporal reasoning is the ability to extract and assimilate temporal information to reconstruct a series of events such that they can be reasoned over to answer questions involving time. Temporal reasoning in the clinical domain is challenging due to specialized medical terms and nomenclature, shorthand notation, fragmented text, a variety of writing styles used by different medical units, redundancy of information that has to be reconciled, and an increased number of temporal references as compared to general domain texts. Work in the area of clinical temporal reasoning has progressed, but the current state-of-the-art still has a ways to go before …

Learning Robot Motion From Creative Human Demonstration, Charles C. Dietzel

Theses and Dissertations

This thesis presents a learning from demonstration framework that enables a robot to learn and perform creative motions from human demonstrations in real-time. In order to satisfy all of the functional requirements for the framework, the developed technique is comprised of two modular components, which integrate together to provide the desired functionality. The first component, called Dancing from Demonstration (DfD), is a kinesthetic learning from demonstration technique. This technique is capable of playing back newly learned motions in real-time, as well as combining multiple learned motions together in a configurable way, either to reduce trajectory error or to generate entirely …

Smart City Management Using Machine Learning Techniques, Mostafa Zaman

Theses and Dissertations

In response to the growing urban population, "smart cities" are designed to improve people's quality of life by implementing cutting-edge technologies. The concept of a "smart city" refers to an effort to enhance a city's residents' economic and environmental well-being via implementing a centralized management system. With the use of sensors and actuators, smart cities can collect massive amounts of data, which can improve people's quality of life and design cities' services. Although smart cities contain vast amounts of data, only a percentage is used due to the noise and variety of the data sources. Information and communication technology (ICT) …

Computational Analysis Of Drug Targets And Prediction Of Protein-Compound Interactions, Sina Ghadermarzi

Theses and Dissertations

Computational prediction of compound-protein interactions generated a substantial amount of interest in the recent years owing to the importance of the knowledge of these interaction for drug discovery and drug repurposing efforts. Research suggests that the currently known drug targets constitute only a fraction of a complete set of drug targets, limiting our ability to identify suitable targets to develop new drugs or to repurpose current drugs for new diseases. These efforts are further thwarted by our limited knowledge of protein-drug (and more generally protein-compound) interactions, where only a subset of drug targets is typically known for the currently used …

Introducing A Real-Time Advanced Eye Movements Analysis Pipeline, Gavindya Jayawardana

Computer Science Faculty Publications

Real-Time Advanced Eye Movements Analysis Pipeline (RAEMAP) is an advanced pipeline to analyze traditional positional gaze measurements as well as advanced eye gaze measurements. The proposed implementation of RAEMAP includes real-time analysis of fixations, saccades, gaze transition entropy, and low/high index of pupillary activity. RAEMAP will also provide visualizations of fixations, fixations on AOIs, heatmaps, and dynamic AOI generation in real-time. This paper outlines the proposed architecture of RAEMAP.

Customer Gaze Estimation In Retail Using Deep Learning, Shashimal Senarath, Primesh Pathirana, Dulani Meedeniya, Sampath Jayarathna

Computer Science Faculty Publications

At present, intelligent computing applications are widely used in different domains, including retail stores. The analysis of customer behaviour has become crucial for the benefit of both customers and retailers. In this regard, the concept of remote gaze estimation using deep learning has shown promising results in analyzing customer behaviour in retail due to its scalability, robustness, low cost, and uninterrupted nature. This study presents a three-stage, three-attention-based deep convolutional neural network for remote gaze estimation in retail using image data. In the first stage, we design a mechanism to estimate the 3D gaze of the subject using image data …

A Novel Framework For Mixed Reality–Based Control Of Collaborative Robot: Development Study, Md. Tanzil Shahria, Md. Samiul Haque Sunny, Md. Ishrak Islam Zarif, Md. Mahafuzur Rahaman Khan, Preet Parag Modi, Sheikh Iqbal Ahamed, Mohammad H. Rahman

Computer Science Faculty Research and Publications

Background:

Applications of robotics in daily life are becoming essential by creating new possibilities in different fields, especially in the collaborative environment. The potentials of collaborative robots are tremendous as they can work in the same workspace as humans. A framework employing a top-notch technology for collaborative robots will surely be worthwhile for further research.

Objective:

This study aims to present the development of a novel framework for the collaborative robot using mixed reality.

Methods:

The framework uses Unity and Unity Hub as a cross-platform gaming engine and project management tool to design the mixed reality interface and digital twin. …

Enhancing Zero-Shot And Few-Shot Text Classification Using Pre-Trained Language Models, Yanan Chen

Theses and Dissertations (Comprehensive)

In recent years, the community of natural language processing (NLP) has seen amazing progress in the development of pre-trained language models (PLMs). The novel paradigm of PLMs does not require labeled data, allowing us to experiment with increased training scale through employing freely available colossal online self-training corpus to push the limits. Language models (LMs), such as GPT, BERT and T5, have achieved high performance on a wide range of NLP tasks. Meanwhile, research on zero-shot and few-shot text classification has received increasing attention. As labelling can be costly and time-consuming, how to perform data augmentation (DA) and enhance the …

Srf Cavity Fault Classification And Prediction At Jefferson Lab, Chris Tennant, Adam Carpenter, Lasitha Vidyaratne, Md. Monibor Rahman, Khan Iftekharuddin

Electrical & Computer Engineering Faculty Publications

Over the last few years several machine learning projects at Jefferson Lab have had a common focus to optimize operation of superconducting RF (SRF) cavities in the Continuous Electron Beam Accelerator Facility (CEBAF). In this talk we highlight work to identify and classify types of faults from C100-type cavities and then to extend those capabilities to provide real-time fault prediction. Early prediction may enable mitigation strategies to prevent some types of faults. In our approach we apply a two-step fault prediction pipeline. In the first step, a model distinguishes between faulty and normal signals. In the second step, signals flagged …

A Channel State Information Based Virtual Mac Spoofing Detector, Peng Jiang, Hongyi Wu, Chunsheng Xin

Electrical & Computer Engineering Faculty Publications

Physical layer security has attracted lots of attention with the expansion of wireless devices to the edge networks in recent years. Due to limited authentication mechanisms, MAC spoofing attack, also known as the identity attack, threatens wireless systems. In this paper, we study a new type of MAC spoofing attack, the virtual MAC spoofing attack, in a tight environment with strong spatial similarities, which can create multiple counterfeits entities powered by the virtualization technologies to interrupt regular services. We develop a system to effectively detect such virtual MAC spoofing attacks via the deep learning method as a countermeasure. …

Runtime Power Allocation Based On Multi-Gpu Utilization In Gamess, Masha Sosonkina, Vaibhav Sundriyal, Jorge Luis Galvez Vallejo

Electrical & Computer Engineering Faculty Publications

To improve the power consumption of parallel applications at the runtime, modern processors provide frequency scaling and power limiting capabilities. In this work, a runtime strategy is proposed to maximize performance under a given power budget by distributing the available power according to the relative GPU utilization. Time series forecasting methods were used to develop workload prediction models that provide accurate prediction of GPU utilization during application execution. Experiments were performed on a multi-GPU computing platform DGX-1 equipped with eight NVIDIA V100 GPUs used for quantum chemistry calculations in the GAMESS package. For a limited power budget, the proposed strategy …

Arithfusion: An Arithmetic Deep Model For Temporal Remote Sensing Image Fusion, Md Reshad Ul Hoque, Jian Wu, Chiman Kwan, Krzysztof Koperski, Jiang Li

Electrical & Computer Engineering Faculty Publications

Different satellite images may consist of variable numbers of channels which have different resolutions, and each satellite has a unique revisit period. For example, the Landsat-8 satellite images have 30 m resolution in their multispectral channels, the Sentinel-2 satellite images have 10 m resolution in the pan-sharp channel, and the National Agriculture Imagery Program (NAIP) aerial images have 1 m resolution. In this study, we propose a simple yet effective arithmetic deep model for multimodal temporal remote sensing image fusion. The proposed model takes both low- and high-resolution remote sensing images at t₁ together with low-resolution images at a …

Facial Landmark Feature Fusion In Transfer Learning Of Child Facial Expressions, Megan A. Witherow, Manar D. Samad, Norou Diawara, Khan M. Iftekharuddin

Electrical & Computer Engineering Faculty Publications

Automatic classification of child facial expressions is challenging due to the scarcity of image samples with annotations. Transfer learning of deep convolutional neural networks (CNNs), pretrained on adult facial expressions, can be effectively finetuned for child facial expression classification using limited facial images of children. Recent work inspired by facial age estimation and age-invariant face recognition proposes a fusion of facial landmark features with deep representation learning to augment facial expression classification performance. We hypothesize that deep transfer learning of child facial expressions may also benefit from fusing facial landmark features. Our proposed model architecture integrates two input branches: a …

The Future Of Ai Accountability In The Financial Markets, Gina-Gail S. Fletcher, Michelle M. Le

Faculty Scholarship

Consumer interaction with the financial market ranges from applying for credit cards, to financing the purchase of a home, to buying and selling securities. And with each transaction, the lender, bank, and brokerage firm are likely utilizing artificial intelligence (AI) behind the scenes to augment their operations. While AI’s ability to process data at high speeds and in large quantities makes it an important tool for financial institutions, it is imperative to be attentive to the risks and limitations that accompany its use. In the context of financial markets, AI’s lack of decision-making transparency, often called the “black box problem,” …