Artificial Intelligence and Robotics Commons^™

An Autoencoder-Based Deep Learning Method For Genotype Imputation, Meng Song, Jonathan Greenbaum, Joseph Luttrell Iv, Weihua Zhou, Chong Wu, Zhe Luo, Chuan Qiu, Lan Juan Zhao, Kuan-Jui Su, Qing Tian, Hui Shen, Huixiao Hong, Ping Gong, Xinghua Shi, Hong-Wen Deng, Chaoyang Zhang

Faculty Publications

Genotype imputation has a wide range of applications in genome-wide association study (GWAS), including increasing the statistical power of association tests, discovering trait-associated loci in meta-analyses, and prioritizing causal variants with fine-mapping. In recent years, deep learning (DL) based methods, such as sparse convolutional denoising autoencoder (SCDA), have been developed for genotype imputation. However, it remains a challenging task to optimize the learning process in DL-based methods to achieve high imputation accuracy. To address this challenge, we have developed a convolutional autoencoder (AE) model for genotype imputation and implemented a customized training loop by modifying the training process with a …

A Monte Carlo Framework For Incremental Improvement Of Simulation Fidelity, Damian Lyons, James Finocchiaro, Misha Novitsky, Chris Korpela

Faculty Publications

Robot software developed in simulation often does not be- have as expected when deployed because the simulation does not sufficiently represent reality - this is sometimes called the `reality gap' problem. We propose a novel algorithm to address the reality gap by injecting real-world experience into the simulation. It is assumed that the robot program (control policy) is developed using simulation, but subsequently deployed on a real system, and that the program includes a performance objective monitor procedure with scalar output. The proposed approach collects simulation and real world observations and builds conditional probability functions. These are used to generate …

Biometrics And An Ai Bill Of Rights, Margaret Hu

Faculty Publications

This Article contends that an informed discussion on an AI Bill of Rights requires grappling with biometric data collection and its integration into emerging AI systems. Biometric AI systems serve a wide range of governmental purposes, including policing, border security and immigration enforcement, and biometric cyberintelligence and biometric-enabled warfare. These systems are increasingly categorized as "high-risk" when deployed in ways that may impact fundamental constitutional rights and human rights. There is growing recognition that high-risk biometric AI systems, such as facial recognition identification, can pose unprecedented challenges to criminal procedure rights. This Article concludes that a failure to recognize these …

A Unified View Of A Human Digital Twin, Michael Miller, Emily Spatz

Faculty Publications

The term human digital twin has recently been applied in many domains, including medical and manufacturing. This term extends the digital twin concept, which has been illustrated to provide enhanced system performance as it combines system models and analyses with real-time measurements for an individual system to improve system maintenance. Human digital twins have the potential to change the practice of human system integration as these systems employ real-time sensing and feedback to tightly couple measurements of human performance, behavior, and environmental influences throughout a product’s life cycle to human models to improve system design and performance. However, as this …

Factored Beliefs For Machine Agents In Decentralized Partially Observable Markov Decision Processes, Joshua A. Lapso, Gilbert L. Peterson

Faculty Publications

A shared mental model (SMM) is a foundational structure in high performing, task-oriented teams and aid humans in determining their teammate's goals and intentions. Higher levels of mental alignment between teammates can reduce the direct dialogue required for team success. For decision-making teams, a transactive memory system (TMS) offers team members a map of specialized knowledge, indicating source of knowledge and the source's credibility. SMM and TMS formulations aid human-agent team performance in their intended team types. However, neither improve team performance with a project team--one that requires both behavioral and knowledge integration. We present a hybrid cognitive model (HCM) …

Evolution Of Combined Arms Tactics In Heterogeneous Multi-Agent Teams, Robert J. Wilson, David W. King, Gilbert L. Peterson

Faculty Publications

Multi-agent systems research is concerned with the emergence of system-level behaviors from relatively simple agent interactions. Multi-agent systems research to date is primarily concerned with systems of homogeneous agents, with member agents both physically and behaviorally identical. Systems of heterogeneous agents with differing physical or behavioral characteristics may be able to accomplish tasks more efficiently than homogeneous teams, via cooperation between mutually complementary agent types. In this article, we compare the performance of homogeneous and heterogeneous teams in combined arms situations. Combined arms theory proposes that the application of heterogeneous forces, en masse, can generate effects far greater than outcomes …

Identification Of Return-Oriented Programming Attacks Using Risc-V Instruction Trace Data, Daniel F. Koranek, Scott R. Graham, Brett J. Borghetti, Wayne C. Henry

Faculty Publications

An increasing number of embedded systems include dedicated neural hardware. To benefit from this specialized hardware, deep learning techniques to discover malware on embedded systems are needed. This effort evaluated candidate machine learning detection techniques for distinguishing exploited from non-exploited RISC-V program behavior using execution traces. We first developed a dataset of execution traces containing Return Oriented Programming (ROP) exploitation on the RISC-V Instruction Set Architecture (ISA) and then developed several deep learning bidirectional Long Short-Term Memory (LSTM) models capable of distinguishing exploited traces from non-exploited traces, each using subsets of features from the execution traces. An objective of this …

Visual Homing For Robot Teams: Do You See What I See?, Damian Lyons, Noah Petzinger

Faculty Publications

Visual homing is a lightweight approach to visual navigation which does not require GPS. It is very attractive for robot platforms with a low computational capacity. However, a limitation is that the stored home location must be initially within the field of view of the robot. Motivated by the increasing ubiquity of camera information we propose to address this line-of-sight limitation by leveraging camera information from other robots and fixed cameras. To home to a location that is not initially within view, a robot must be able to identify a common visual landmark with another robot that can be used …

Evaluation Of Deep Neural Network Prospr For Accurate Protein Distance Predictions On Casp14 Targets, Jacob A. Stern, Bryce Eric Hedelius, Olivia Fisher, Wendy M. Billings, Dennis Della Corte

Faculty Publications

The field of protein structure prediction has recently been revolutionized through the introduction of deep learning. The current state-of-the-art tool AlphaFold2 can predict highly accurate structures; however, it has a prohibitively long inference time for applications that require the folding of hundreds of sequences. The prediction of protein structure annotations, such as amino acid distances, can be achieved at a higher speed with existing tools, such as the ProSPr network. Here, we report on important updates to the ProSPr network, its performance in the recent Critical Assessment of Techniques for Protein Structure Prediction (CASP14) competition, and an evaluation of its …

Cognition-Enhanced Machine Learning For Better Predictions With Limited Data, Florian Sense, Ryan Wood, Michael G. Collins, Joshua Fiechter, Aihua W. Wood, Michael Krusmark, Tiffany Jastrzembski, Christopher W. Myers

Faculty Publications

The fields of machine learning (ML) and cognitive science have developed complementary approaches to computationally modeling human behavior. ML's primary concern is maximizing prediction accuracy; cognitive science's primary concern is explaining the underlying mechanisms. Cross-talk between these disciplines is limited, likely because the tasks and goals usually differ. The domain of e-learning and knowledge acquisition constitutes a fruitful intersection for the two fields’ methodologies to be integrated because accurately tracking learning and forgetting over time and predicting future performance based on learning histories are central to developing effective, personalized learning tools. Here, we show how a state-of-the-art ML model can …

Year-Independent Prediction Of Food Insecurity Using Classical & Neural Network Machine Learning Methods, Caleb Christiansen, Torrey J. Wagner, Brent Langhals

Faculty Publications

Current food crisis predictions are developed by the Famine Early Warning System Network, but they fail to classify the majority of food crisis outbreaks with model metrics of recall (0.23), precision (0.42), and f1 (0.30). In this work, using a World Bank dataset, classical and neural network (NN) machine learning algorithms were developed to predict food crises in 21 countries. The best classical logistic regression algorithm achieved a high level of significance (p < 0.001) and precision (0.75) but was deficient in recall (0.20) and f1 (0.32). Of particular interest, the classical algorithm indicated that the vegetation index and the food price index were both positively correlated with food crises. A novel method for performing an iterative multidimensional hyperparameter search is presented, which resulted in significantly improved performance when applied to this dataset. Four iterations were conducted, which resulted in excellent 0.96 for metrics of precision, recall, and f1. Due to this strong performance, the food crisis year was removed from the dataset to prevent immediate extrapolation when used on future data, and the modeling process was repeated. The best “no year” model metrics remained strong, achieving ≥0.92 for recall, precision, and f1 while meeting a 10% f1 overfitting threshold on the test (0.84) and holdout (0.83) datasets. The year-agnostic neural network model represents a novel approach to classify food crises and outperforms current food crisis prediction efforts.

The Whole Is Greater Than Its Parts: Ensembling Improves Protein Contact Prediction, Wendy M. Billings, Connor J. Morris, Dennis Della Corte

Faculty Publications

The prediction of amino acid contacts from protein sequence is an important problem, as protein contacts are a vital step towards the prediction of folded protein structures. We propose that a powerful concept from deep learning, called ensembling, can increase the accuracy of protein contact predictions by combining the outputs of different neural network models. We show that ensembling the predictions made by different groups at the recent Critical Assessment of Protein Structure Prediction (CASP13) outperforms all individual groups. Further, we show that contacts derived from the distance predictions of three additional deep neural networks—AlphaFold, trRosetta, and ProSPr—can be substantially …

A Physics-Based Machine Learning Study Of The Behavior Of Interstitial Helium In Single Crystal W–Mo Binary Alloys, Adib J. Samin

Faculty Publications

In this work, the behavior of dilute interstitial helium in W–Mo binary alloys was explored through the application of a first principles-informed neural network (NN) in order to study the early stages of helium-induced damage and inform the design of next generation materials for fusion reactors. The neural network (NN) was trained using a database of 120 density functional theory (DFT) calculations on the alloy. The DFT database of computed solution energies showed a linear dependence on the composition of the first nearest neighbor metallic shell. This NN was then employed in a kinetic Monte Carlo simulation, which took into …

Using Taint Analysis And Reinforcement Learning (Tarl) To Repair Autonomous Robot Software, Damian Lyons, Saba Zahra

Faculty Publications

It is important to be able to establish formal performance bounds for autonomous systems. However, formal verification techniques require a model of the environment in which the system operates; a challenge for autonomous systems, especially those expected to operate over longer timescales. This paper describes work in progress to automate the monitor and repair of ROS-based autonomous robot software written for an a-priori partially known and possibly incorrect environment model. A taint analysis method is used to automatically extract the data-flow sequence from input topic to publish topic, and instrument that code. A unique reinforcement learning approximation of MDP utility …

Keeping Ai Under Observation: Anticipated Impacts On Physicians' Standard Of Care, Iria Giuffrida, Taylor Treece

Faculty Publications

As Artificial Intelligence (AI) tools become increasingly present across industries, concerns have started to emerge as to their impact on professional liability. Specifically, for the medical industry--in many ways an inherently "risky" business--hospitals and physicians have begun evaluating the impact of Al tools on their professional malpractice risk. This Essay seeks to address that question, zooming in on how AI may affect physicians' standard of care for medical malpractice claims.

A New Ectotherm 3d Tracking And Behavior Analytics System Using A Depth-Based Approach With Color Validation, With Preliminary Data On Kihansi Spray Toad (Nectophrynoides Asperginis) Activity, Philip Bal, Damian Lyons, Avishai Shuter

Faculty Publications

The Kihansi spray toad (Nectophrynoides asperginis), classified as Extinct in the Wild by the IUCN, is being bred at the Wildlife Conservation Society’s (WCS) Bronx Zoo as part of an effort to successfully reintroduce the species into the wild. Thousands of toads live at the Bronx Zoo presenting an opportunity to learn more about their behaviors for the first time, at scale. It is impractical to perform manual observations for long periods of time. This paper reports on the development of a RGB-D tracking and analytics approach that allows researchers to accurately and efficiently gather information about the toads’ behavior. …

Cyber-Physical Security With Rf Fingerprint Classification Through Distance Measure Extensions Of Generalized Relevance Learning Vector Quantization, Trevor J. Bihl, Todd J. Paciencia, Kenneth W. Bauer Jr., Michael A. Temple

Faculty Publications

Radio frequency (RF) fingerprinting extracts fingerprint features from RF signals to protect against masquerade attacks by enabling reliable authentication of communication devices at the “serial number” level. Facilitating the reliable authentication of communication devices are machine learning (ML) algorithms which find meaningful statistical differences between measured data. The Generalized Relevance Learning Vector Quantization-Improved (GRLVQI) classifier is one ML algorithm which has shown efficacy for RF fingerprinting device discrimination. GRLVQI extends the Learning Vector Quantization (LVQ) family of “winner take all” classifiers that develop prototype vectors (PVs) which represent data. In LVQ algorithms, distances are computed between exemplars and PVs, and …

A Monte Carlo Approach To Closing The Reality Gap, Damian Lyons, James Finocchiaro, Michael Novitzky, Christopher Korpela

Faculty Publications

We propose a novel approach to the ’reality gap’ problem, i.e., modifying a robot simulation so that its performance becomes more similar to observed real world phenomena. This problem arises whether the simulation is being used by human designers or in an automated policy development mechanism. We expect that the program/policy is developed using simulation, and subsequently deployed on a real system. We further assume that the program includes a monitor procedure with scalar output to determine when it is achieving its performance objectives. The proposed approach collects simulation and real world observations and builds conditional probability functions. These are …

Liability For Ai Decision-Making: Some Legal And Ethical Considerations, Iria Giuffrida

Faculty Publications

No abstract provided.

Sequence Pattern Mining With Variables, James S. Okolica, Gilbert L. Peterson, Robert F. Mills, Michael R. Grimaila

Faculty Publications

Sequence pattern mining (SPM) seeks to find multiple items that commonly occur together in a specific order. One common assumption is that all of the relevant differences between items are captured through creating distinct items, e.g., if color matters then the same item in two different colors would have two items created, one for each color. In some domains, that is unrealistic. This paper makes two contributions. The first extends SPM algorithms to allow item differentiation through attribute variables for domains with large numbers of items, e.g, by having one item with a variable with a color attribute rather than …

A Macro-Level Order Metric For Self-Organizing Adaptive Systems, David W. King, Gilbert L. Peterson

Faculty Publications

Analyzing how agent interactions affect macro-level self-organized behaviors can yield a deeper understanding of how complex adaptive systems work. The dynamic nature of complex systems makes it difficult to determine if, or when, a system has reached a state of equilibrium or is about to undergo a major transition reflecting the appearance of self-organized states. Using the notion of local neighborhood entropy, this paper presents a metric for evaluating the macro-level order of a system. The metric is tested in two dissimilar complex adaptive systems with self-organizing properties: An autonomous swarm searching for multiple dynamic targets and Conway's Game of …

A Legal Perspective On The Trials And Tribulations Of Ai: How Artificial Intelligence, The Internet Of Things, Smart Contracts, And Other Technologies Will Affect The Law, Iria Giuffrida, Fredric Lederer, Nicolas Vermeys

Faculty Publications

No abstract provided.

Does The Test Work? Evaluating A Web-Based Language Placement Test, Avizia Long, Sun-Young Shin, Kimberly Geeslin, Erik Willis

Faculty Publications

In response to the need for examples of test validation from which everyday language programs can benefit, this paper reports on a study that used Bachman’s (2005) assessment use argument (AUA) framework to examine evidence to support claims made about the intended interpretations and uses of scores based on a new web-based Spanish language placement test. The test, which consisted of 100 items distributed across five item types (sound discrimination, grammar, listening comprehension, reading comprehension, and vocabulary), was tested with 2,201 incoming first-year and transfer students at a large, Midwestern public university. Analyses of internal consistency and validity revealed the …

Formal Performance Guarantees For An Approach To Human In The Loop Robot Missions, Damian Lyons, Ron Arkin, Shu Jiang, Matt O'Brien, Feng Tang, Peng Tang

Faculty Publications

Abstract— A key challenge in the automatic verification of robot mission software, especially critical mission software, is to be able to effectively model the performance of a human operator and factor that into the formal performance guarantees for the mission. We present a novel approach to modelling the skill level of the operator and integrating it into automatic verification using a linear Gaussians model parameterized by experimental calibration. Our approach allows us to model different skill levels directly in terms of the behavior of the lumped, robot plus operator, system.

Using MissionLab and VIPARS (a behavior-based robot mission verification …

An Approach To Robust Homing With Stereovision, Fuqiang Fu, Damian Lyons

Faculty Publications

Visual Homing is a bioinspired approach to robot navigation which can be fast and uses few assumptions. However, visual homing in a cluttered and unstructured outdoor environment offers several challenges to homing methods that have been developed for primarily indoor environments. One issue is that any current image during homing may be tilted with respect to the home image. The second is that moving through a cluttered scene during homing may cause obstacles to interfere between the home scene and location and the current scene and location. In this paper, we introduce a robust method to improve a previous developed …

Performance Verification For Robot Missions In Uncertain Environments, Damian Lyons, Ron Arkin, Shu Jiang, Matt O'Brien, Feng Tang, Peng Tang

Faculty Publications

Abstract—Certain robot missions need to perform predictably in a physical environment that may have significant uncertainty. One approach is to leverage automatic software verification techniques to establish a performance guarantee. The addition of an environment model and uncertainty in both program and environment, however, means the state-space of a model-checking solution to the problem can be prohibitively large. An approach based on behavior-based controllers in a process-algebra framework that avoids state-space combinatorics is presented here. In this approach, verification of the robot program in the uncertain environment is reduced to a filtering problem for a Bayesian Network. Validation results …

Establishing A-Priori Performance Guarantees For Robot Missions That Include Localization Software, Damian Lyons, Ron Arkin, Shu Jiang, Matt O'Brien, Feng Tang, Peng Tang

Faculty Publications

One approach to determining whether an automated system is performing correctly is to monitor its performance, signaling when the performance is not acceptable; another approach is to automatically analyze the possible behaviors of the system a-priori and determine performance guarantees. Thea authors have applied this second approach to automatically derive performance guarantees for behaviorbased, multi-robot critical mission software using an innovative approach to formal verification for robotic software. Localization and mapping algorithms can allow a robot to navigate well in an unknown environment. However, whether such algorithms enhance any specific robot mission is currently a matter for empirical validation. Several …

Formal Performance Guarantees For Behavior-Based Localization Missions, Damian Lyons, Ron Arkin, Shu Jiang, Matt O'Brien, Feng Tang, Peng Tang

Faculty Publications

Abstract— Localization and mapping algorithms can allow a robot to navigate well in an unknown environment. However, whether such algorithms enhance any specific robot mission is currently a matter for empirical validation. In this paper we apply our MissionLab/VIPARS mission design and verification approach to an autonomous robot mission that uses probabilistic localization software.

Two approaches to modeling probabilistic localization for verification are presented: a high-level approach, and a sample-based approach which allows run-time code to be embedded in verification. Verification and experimental validation results are presented for two different missions, each using each method, demonstrating the accuracy …

Landmark Detection With Surprise Saliency Using Convolutional Neural Networks, Feng Tang, Damian Lyons, Daniel Leeds

Faculty Publications

Abstract—Landmarks can be used as reference to enable people or robots to localize themselves or to navigate in their environment. Automatic definition and extraction of appropriate landmarks from the environment has proven to be a challenging task when pre-defined landmarks are not present. We propose a novel computational model of automatic landmark detection from a single image without any pre-defined landmark database. The hypothesis is that if an object looks abnormal due to its atypical scene context (what we call surprise saliency), it then may be considered as a good landmark because it is unique and easy to spot by …

Machine Learning Nuclear Detonation Features, Daniel T. Schmitt, Gilbert L. Peterson

Faculty Publications

Nuclear explosion yield estimation equations based on a 3D model of the explosion volume will have a lower uncertainty than radius based estimation. To accurately collect data for a volume model of atmospheric explosions requires building a 3D representation from 2D images. The majority of 3D reconstruction algorithms use the SIFT (scale-invariant feature transform) feature detection algorithm which works best on feature-rich objects with continuous angular collections. These assumptions are different from the archive of nuclear explosions that have only 3 points of view. This paper reduces 300 dimensions derived from an image based on Fourier analysis and five edge …