Artificial Intelligence and Robotics Commons^™

Towards Improving Accuracy And Interpretability Of Deep Learning Based On Satellite Image Classification, Yamile Patino Vargas

Dissertations and Theses

ABSTRACT

The study of satellite images provides a way to monitor changes in the surface of the Earth and the atmosphere. Convolutional Neural Networks (CNN) have shown accurate results in solving practical problems in multiple fields. Some of the more recognized fields using CNNs are satellite imagery processing, medicine, communication, transportation, and computer vision. Despite the success of CNNs, there remains a need to explain the network predictions further and understand what the network is determining as valuable information.

There are several frameworks and methodologies developed to explain how CNNs predict outputs and what their internal representations are [1, 4, …

Personality Recognition For Deception Detection, Guozhen An

Dissertations, Theses, and Capstone Projects

Personality aims at capturing stable individual characteristics, typically measurable in quantitative terms, that explain and predict observable behavioral differences. Personality has been proved to be very useful in many life outcomes, and there has been huge interests on predicting personality automatically. Previously, there are tremendous amount of approaches successfully predicting personality. However, most previous research on personality detection has used personality scores assigned by annotators based solely on the text or audio clip, and found that predicting self-reported personality is a much more difficult task than predicting observer-report personality. In our study, we will demonstrate how to accurately detect self-reported …

Enhancing 3d Visual Odometry With Single-Camera Stereo Omnidirectional Systems, Carlos A. Jaramillo

Dissertations, Theses, and Capstone Projects

We explore low-cost solutions for efficiently improving the 3D pose estimation problem of a single camera moving in an unfamiliar environment. The visual odometry (VO) task -- as it is called when using computer vision to estimate egomotion -- is of particular interest to mobile robots as well as humans with visual impairments. The payload capacity of small robots like micro-aerial vehicles (drones) requires the use of portable perception equipment, which is constrained by size, weight, energy consumption, and processing power. Using a single camera as the passive sensor for the VO task satisfies these requirements, and it motivates the …

Adaptation And Augmentation: Towards Better Rescoring Strategies For Automatic Speech Recognition And Spoken Term Detection, Min Ma

Dissertations, Theses, and Capstone Projects

Selecting the best prediction from a set of candidates is an essential problem for many spoken language processing tasks, including automatic speech recognition (ASR) and spoken keyword spotting (KWS). Generally, the selection is determined by a confidence score assigned to each candidate. Calibrating these confidence scores (i.e., rescoring them) could make better selections and improve the system performance. This dissertation focuses on using tailored language models to rescore ASR hypotheses as well as keyword search results for ASR-based KWS.

This dissertation introduces three kinds of rescoring techniques: (1) Freezing most model parameters while fine-tuning the output layer in order to …

Multimodal Depression Detection: An Investigation Of Features And Fusion Techniques For Automated Systems, Michelle Renee Morales

Dissertations, Theses, and Capstone Projects

Depression is a serious illness that affects a large portion of the world’s population. Given the large effect it has on society, it is evident that depression is a serious health issue. This thesis evaluates, at length, how technology may aid in assessing depression. We present an in-depth investigation of features and fusion techniques for depression detection systems. We also present OpenMM: a novel tool for multimodal feature extraction. Lastly, we present novel techniques for multimodal fusion. The contributions of this work add considerably to our knowledge of depression detection systems and have the potential to improve future systems by …

On The Use Of Semantic-Based Aig To Automatically Generate Programming Exercises, Laura Zavala, Benito Mendoza

Publications and Research

In introductory programming courses, proficiency is typically achieved through substantial practice in the form of relatively small assignments and quizzes. Unfortunately, creating programming assignments and quizzes is both, time-consuming and error-prone. We use Automatic Item Generation (AIG) in order to address the problem of creating numerous programming exercises that can be used for assignments or quizzes in introductory programming courses. AIG is based on the use of test-item templates with embedded variables and formulas which are resolved by a computer program with actual values to generate test-items. Thus, hundreds or even thousands of test-items can be generated with a single …

Vision-Based Assistive Indoor Localization, Feng Hu

Dissertations, Theses, and Capstone Projects

An indoor localization system is of significant importance to the visually impaired in their daily lives by helping them localize themselves and further navigate an indoor environment. In this thesis, a vision-based indoor localization solution is proposed and studied with algorithms and their implementations by maximizing the usage of the visual information surrounding the users for an optimal localization from multiple stages. The contributions of the work include the following: (1) Novel combinations of a daily-used smart phone with a low-cost lens (GoPano) are used to provide an economic, portable, and robust indoor localization service for visually impaired people. (2) …

Multimodal Sensing And Data Processing For Speaker And Emotion Recognition Using Deep Learning Models With Audio, Video And Biomedical Sensors, Farnaz Abtahi

Dissertations, Theses, and Capstone Projects

The focus of the thesis is on Deep Learning methods and their applications on multimodal data, with a potential to explore the associations between modalities and replace missing and corrupt ones if necessary. We have chosen two important real-world applications that need to deal with multimodal data: 1) Speaker recognition and identification; 2) Facial expression recognition and emotion detection.

The first part of our work assesses the effectiveness of speech-related sensory data modalities and their combinations in speaker recognition using deep learning models. First, the role of electromyography (EMG) is highlighted as a unique biometric sensor in improving audio-visual speaker …

Object Localization, Segmentation, And Classification In 3d Images, Allan Zelener

Dissertations, Theses, and Capstone Projects

We address the problem of identifying objects of interest in 3D images as a set of related tasks involving localization of objects within a scene, segmentation of observed object instances from other scene elements, classifying detected objects into semantic categories, and estimating the 3D pose of detected objects within the scene. The increasing availability of 3D sensors motivates us to leverage large amounts of 3D data to train machine learning models to address these tasks in 3D images. Leveraging recent advances in deep learning has allowed us to develop models capable of addressing these tasks and optimizing these tasks jointly …

Gradient Estimation For Attractor Networks, Thomas Flynn

Dissertations, Theses, and Capstone Projects

It has been hypothesized that neural network models with cyclic connectivity may be more powerful than their feed-forward counterparts. This thesis investigates this hypothesis in several ways. We study the gradient estimation and optimization procedures for several variants of these networks. We show how the convergence of the gradient estimation procedures are related to the properties of the networks. Then we consider how to tune the relative rates of gradient estimation and parameter adaptation to ensure successful optimization in these models. We also derive new gradient estimators for stochastic models. First, we port the forward sensitivity analysis method to the …

Deep Convolutional Neural Networks Enable Discrimination Of Heterogeneous Digital Pathology Images, Pegah Khosravi, Ehsan Kazemi, Marcin Imielinski, Olivier Elemento, Iman Hajirasouliha

Publications and Research

Pathological evaluation of tumor tissue is pivotal for diagnosis in cancer patients and automated image analysis approaches have great potential to increase precision of diagnosis and help reduce human error.

In this study, we utilize several computational methods based on convolutional neural networks (CNN) and build a stand-alone pipeline to effectively classify different histopathology images across different types of cancer.

In particular, we demonstrate the utility of our pipeline to discriminate between two subtypes of lung cancer, four biomarkers of bladder cancer, and five biomarkers of breast cancer. In addition, we apply our pipeline to discriminate among four immunohistochemistry …

The Ability Of Different Imputation Methods To Preserve The Significant Genes And Pathways In Cancer, Rosa Aghdam, Taban Baghfalaki, Pegah Khosravi, Elnaz Saberi Ansari

Publications and Research

Deciphering important genes and pathways from incomplete gene expression data could facilitate a better understanding of cancer. Different imputation methods can be applied to estimate the missing values. In our study, we evaluated various imputation methods for their performance in preserving significant genes and pathways. In the first step, 5% genes are considered in random for two types of ignorable and non-ignorable missingness mechanisms with various missing rates. Next, 10 well-known imputation methods were applied to the complete datasets. The significance analysis of microarrays (SAM) method was applied to detect the significant genes in rectal and lung cancers to showcase …

Ethics And Bias In Machine Learning: A Technical Study Of What Makes Us “Good”, Ashley Nicole Shadowen

Student Theses

The topic of machine ethics is growing in recognition and energy, but bias in machine learning algorithms outpaces it to date. Bias is a complicated term with good and bad connotations in the field of algorithmic prediction making. Especially in circumstances with legal and ethical consequences, we must study the results of these machines to ensure fairness. This paper attempts to address ethics at the algorithmic level of autonomous machines. There is no one solution to solving machine bias, it depends on the context of the given system and the most reasonable way to avoid biased decisions while maintaining the …

On Improvised Music, Computational Creativity And Human-Becoming, Arto Artinian, Adam James Wilson

Publications and Research

Music improvisation is an act of human-becoming: of self-expression—an articulation of histories and memories that have molded its participants—and of exploration—a search for unimagined structures that break with the stale norms of majoritarian culture. Given that the former objective may inhibit the latter, we propose an integration of human musical improvisers and deliberately flawed creative software agents that are designed to catalyze the development of human-ratified minoritarian musical structures.

Machine Learning Approach To Retrieving Physical Variables From Remotely Sensed Data, Fazlul Shahriar

Dissertations, Theses, and Capstone Projects

Scientists from all over the world make use of remotely sensed data from hundreds of satellites to better understand the Earth. However, physical measurements from an instrument is sometimes missing either because the instrument hasn't been launched yet or the design of the instrument omitted a particular spectral band. Measurements received from the instrument may also be corrupt due to malfunction in the detectors on the instrument. Fortunately, there are machine learning techniques to estimate the missing or corrupt data. Using these techniques we can make use of the available data to its full potential.

We present work on four …

Machine Learning Algorithms For Automated Satellite Snow And Sea Ice Detection, George Bonev

Dissertations, Theses, and Capstone Projects

The continuous mapping of snow and ice cover, particularly in the arctic and poles, are critical to understanding the earth and atmospheric science. Much of the world's sea ice and snow covers the most inhospitable places, making measurements from satellite-based remote sensors essential. Despite the wealth of data from these instruments many challenges remain. For instance, remote sensing instruments reside on-board different satellites and observe the earth at different portions of the electromagnetic spectrum with different spatial footprints. Integrating and fusing this information to make estimates of the surface is a subject of active research.

In response to these challenges, …

Travel Mode Identification With Smartphone Sensors, Xing Su

Dissertations, Theses, and Capstone Projects

Personal trips in a modern urban society typically involve multiple travel modes. Recognizing a traveller's transportation mode is not only critical to personal context-awareness in related applications, but also essential to urban traffic operations, transportation planning, and facility design. While the state of the art in travel mode recognition mainly relies on large-scale infrastructure-based fixed sensors or on individuals' GPS devices, the emergence of the smartphone provides a promising alternative with its ever-growing computing, networking, and sensing powers. In this thesis, we propose new algorithms for travel mode identification using smartphone sensors. The prototype system is built upon the latest …

Solving Algorithmic Problems In Finitely Presented Groups Via Machine Learning, Jonathan Gryak

Dissertations, Theses, and Capstone Projects

Machine learning and pattern recognition techniques have been successfully applied to algorithmic problems in free groups. In this dissertation, we seek to extend these techniques to finitely presented non-free groups, in particular to polycyclic and metabelian groups that are of interest to non-commutative cryptography.

As a prototypical example, we utilize supervised learning methods to construct classifiers that can solve the conjugacy decision problem, i.e., determine whether or not a pair of elements from a specified group are conjugate. The accuracies of classifiers created using decision trees, random forests, and N-tuple neural network models are evaluated for several non-free groups. …

Comparing Tensorflow Deep Learning Performance Using Cpus, Gpus, Local Pcs And Cloud, John Lawrence, Jonas Malmsten, Andrey Rybka, Daniel A. Sabol, Ken Triplin

Publications and Research

Deep learning is a very computational intensive task. Traditionally GPUs have been used to speed-up computations by several orders of magnitude. TensorFlow is a deep learning framework designed to improve performance further by running on multiple nodes in a distributed system. While TensorFlow has only been available for a little over a year, it has quickly become the most popular open source machine learning project on GitHub. The open source version of TensorFlow was originally only capable of running on a single node while Google’s proprietary version only was capable of leveraging distributed systems. This has now changed. In this …

2d Vector Map And Database Design For Indoor Assisted Navigation, Luciano Caraciolo Albuquerque

Dissertations and Theses

In this paper we implemented a 2D Vector Map, map editor and Database design intended to provide an efficient way to convert cad files from indoor environments to a set of vectors representing hallways, doors, exits, elevators, and other entities embedded in a floor plan, and save them in a database for use by other applications, such as assisted navigation for blind people.

A graphical application as developed in C++ to allow the user to input a CAD DXF file, process the file to automatically obtain nodes and edges, and save the nodes and edges to a database for posterior …

Rationality, Parapsychology, And Artificial Intelligence In Military And Intelligence Research By The United States Government In The Cold War, Guy M. Lomeo

Theses and Dissertations

A study analyzing the roles of rationality, parapsychology, and artificial intelligence in military and intelligence research by the United States Government in the Cold War. An examination of the methodology behind the decisions to pursue research in two fields that were initially considered irrational.

A Study Of The Impact Of Interaction Mechanisms And Population Diversity In Evolutionary Multiagent Systems, Sadat U. Chowdhury

Dissertations, Theses, and Capstone Projects

In the Evolutionary Computation (EC) research community, a major concern is maintaining optimal levels of population diversity. In the Multiagent Systems (MAS) research community, a major concern is implementing effective agent coordination through various interaction mechanisms. These two concerns coincide when one is faced with Evolutionary Multiagent Systems (EMAS).

This thesis demonstrates a methodology to study the relationship between interaction mechanisms, population diversity, and performance of an evolving multiagent system in a dynamic, real-time, and asynchronous environment. An open sourced extensible experimentation platform is developed that allows plug-ins for evolutionary models, interaction mechanisms, and genotypical encoding schemes beyond the one …

A Genetic Algorithmic Approach To Automated Auction Mechanism Design, Jinzhong Niu, Simon Parsons

Publications and Research

In this paper, we present a genetic algorithmic approach to automated auction mechanism design in the context of \cat games. This is a follow-up to one piece of our prior work in the domain, the reinforcement learning-based grey-box approach. Our experiments show that given the same search space the grey-box approach is able to produce better auction mechanisms than the genetic algorithmic approach. The comparison can also shed light on the design and evaluation of similar search solutions to other domain problems.

Data-Driven Synthesis And Evaluation Of Syntactic Facial Expressions In American Sign Language Animation, Hernisa Kacorri

Dissertations, Theses, and Capstone Projects

Technology to automatically synthesize linguistically accurate and natural-looking animations of American Sign Language (ASL) would make it easier to add ASL content to websites and media, thereby increasing information accessibility for many people who are deaf and have low English literacy skills. State-of-art sign language animation tools focus mostly on accuracy of manual signs rather than on the facial expressions. We are investigating the synthesis of syntactic ASL facial expressions, which are grammatically required and essential to the meaning of sentences. In this thesis, we propose to: (1) explore the methodological aspects of evaluating sign language animations with facial expressions, …

Soft Robotic Grippers For Biological Sampling On Deep Reefs, Kevin C. Galloway, Kaitlyn P. Becker, Brennan Phillips, Jordan Kirby, Stephen Licht, Dan Tchernov, Robert J. Wood, David F. Gruber

Publications and Research

This article presents the development of an underwater gripper that utilizes soft robotics technology to delicately manipulate and sample fragile species on the deep reef. Existing solutions for deep sea robotic manipulation have historically been driven by the oil industry, resulting in destructive interactions with undersea life. Soft material robotics relies on compliant materials that are inherently impedance matched to natural environments and to soft or fragile organisms. We demonstrate design principles for soft robot end effectors, bench-top characterization of their grasping performance, and conclude by describing in situ testing at mesophotic depths. The result is the first use of …

An Approach To Automatic Detection Of Suspicious Individuals In A Crowd, Satabdi Mukherjee

Dissertations and Theses

This paper describes an approach to identify individuals with suspicious objects in a crowd. It is based on a well-known image retrieval problem as applied to mobile visual search. In many cases, the process of building a hierarchical tree uses k-means clustering followed by geometric verification. However, the number of clusters is not known in advance, and sometimes it is randomly generated. This may lead to a congested clustering which can cause problems in grouping large real-time data. To overcome this problem we have applied the Indian Buffet stochastic process approach in this paper to the clustering problem. We present …

Factororacle: An Extensible Max External For Investigating Applications Of The Factor Oracle Automaton In Real-Time Music Improvisation, Adam James Wilson

Publications and Research

There are several extant software systems designed to generate music in real-time using a factor oracle automaton constructed from the musical input of a human improvisor. The impetus for the design of the factorOracle external is neither a desire to supersede these systems nor introduce novel algorithms for traversing the oracle, but rather to provide a fast, canonical interface for the automaton in Cycling74’s Max and, in future iterations, the Pure Data programming environment. Technical features of the factorOracle software are introduced here.

Inferring Interaction Type In Gene Regulatory Networks Using Co-Expression Data, Pegah Khosravi, Vahid H. Gazestani, Leila Pirhaji, Brian Law, Mehdi Sadeghi, Bahram Goliaei, Gary D. Bader

Publications and Research

Background

Knowledge of interaction types in biological networks is important for understanding the functional organization of the cell. Currently information-based approaches are widely used for inferring gene regulatory interactions from genomics data, such as gene expression profiles; however, these approaches do not provide evidence about the regulation type (positive or negative sign) of the interaction.

Results

This paper describes a novel algorithm, “Signing of Regulatory Networks” (SIREN), which can infer the regulatory type of interactions in a known gene regulatory network (GRN) given corresponding genome-wide gene expression data. To assess our new approach, we applied it to three different benchmark …

Automated Classification Of Argument Stance In Student Essays: A Linguistically Motivated Approach With An Application For Supporting Argument Summarization, Adam Robert Faulkner

Dissertations, Theses, and Capstone Projects

This study describes a set of document- and sentence-level classification models designed to automate the task of determining the argument stance (for or against) of a student argumentative essay and the task of identifying any arguments in the essay that provide reasons in support of that stance. A suggested application utilizing these models is presented which involves the automated extraction of a single-sentence summary of an argumentative essay. This summary sentence indicates the overall argument stance of the essay from which the sentence was extracted and provides a representative argument in support of that stance.

A novel set …

Cross-Talk: A Shared Parameter Space For Gesturally Extended Human/Machine Improvisation, William Brent, Adam James Wilson

Publications and Research

This paper describes Cross-talk, a piece of music and performance system for two instruments augmented with infrared motion-tracking capability, and an artificial software improviser. Cross-talk was commissioned by the Ammerman Center for Arts and Technology at Connecticut College, for the 13th Biennial Symposium on Arts and Technology. The work is part of an ongoing collaboration focused on developing integrated hardware and software performance systems to extend the timbral and expressive capabilities of traditional musical instruments and to generate musical structure in response to information retrieved from human performers in real-time. Artistic motivations and prior related work are presented here, along …