Artificial Intelligence and Robotics Commons^™

Using Taint Analysis And Reinforcement Learning (Tarl) To Repair Autonomous Robot Software, Damian Lyons, Saba Zahra

Faculty Publications

It is important to be able to establish formal performance bounds for autonomous systems. However, formal verification techniques require a model of the environment in which the system operates; a challenge for autonomous systems, especially those expected to operate over longer timescales. This paper describes work in progress to automate the monitor and repair of ROS-based autonomous robot software written for an a-priori partially known and possibly incorrect environment model. A taint analysis method is used to automatically extract the data-flow sequence from input topic to publish topic, and instrument that code. A unique reinforcement learning approximation of MDP utility …

Functional Object-Oriented Network: A Knowledge Representation For Service Robotics, David Andrés Paulius Ramos

USF Tampa Graduate Theses and Dissertations

In this dissertation, we discuss our work behind the development of the functional object-oriented network (abbreviated as FOON), a graphical knowledge representation for robotic manipulation and understanding of its own actions and (potentially) the intentions of humans in the household. Based on the theory of affordance, this representation captures manipulations and their effects on actions through the coupling of object and motion nodes as fundamental learning units known as functional units. The activities currently represented in FOON are cooking related, but this representation can be extended to other activities that involve manipulation of objects which result in observable changes of …

A New Ectotherm 3d Tracking And Behavior Analytics System Using A Depth-Based Approach With Color Validation, With Preliminary Data On Kihansi Spray Toad (Nectophrynoides Asperginis) Activity, Philip Bal, Damian Lyons, Avishai Shuter

Faculty Publications

The Kihansi spray toad (Nectophrynoides asperginis), classified as Extinct in the Wild by the IUCN, is being bred at the Wildlife Conservation Society’s (WCS) Bronx Zoo as part of an effort to successfully reintroduce the species into the wild. Thousands of toads live at the Bronx Zoo presenting an opportunity to learn more about their behaviors for the first time, at scale. It is impractical to perform manual observations for long periods of time. This paper reports on the development of a RGB-D tracking and analytics approach that allows researchers to accurately and efficiently gather information about the toads’ behavior. …

A Monte Carlo Approach To Closing The Reality Gap, Damian Lyons, James Finocchiaro, Michael Novitzky, Christopher Korpela

Faculty Publications

We propose a novel approach to the ’reality gap’ problem, i.e., modifying a robot simulation so that its performance becomes more similar to observed real world phenomena. This problem arises whether the simulation is being used by human designers or in an automated policy development mechanism. We expect that the program/policy is developed using simulation, and subsequently deployed on a real system. We further assume that the program includes a monitor procedure with scalar output to determine when it is achieving its performance objectives. The proposed approach collects simulation and real world observations and builds conditional probability functions. These are …

Facial Action Unit Detection With Deep Convolutional Neural Networks, Siddhesh Padwal

Electronic Theses and Dissertations

The facial features are the most important tool to understand an individual's state of mind. Automated recognition of facial expressions and particularly Facial Action Units defined by Facial Action Coding System (FACS) is challenging research problem in the field of computer vision and machine learning. Researchers are working on deep learning algorithms to improve state of the art in the area. Automated recognition of facial action units has man applications ranging from developmental psychology to human robot interface design where companies are using this technology to improve their consumer devices (like unlocking phone) and for entertainment like FaceApp. Recent studies …

A Comprehensive And Modular Robotic Control Framework For Model-Less Control Law Development Using Reinforcement Learning For Soft Robotics, Charles Sullivan

Open Access Theses & Dissertations

Soft robotics is a growing field in robotics research. Heavily inspired by biological systems, these robots are made of softer, non-linear, materials such as elastomers and are actuated using several novel methods, from fluidic actuation channels to shape changing materials such as electro-active polymers. Highly non-linear materials make modeling difficult, and sensors are still an area of active research. These issues have rendered typical control and modeling techniques often inadequate for soft robotics. Reinforcement learning is a branch of machine learning that focuses on model-less control by mapping states to actions that maximize a specific reward signal. Reinforcement learning has …

Comparison Of Object Detection And Patch-Based Classification Deep Learning Models On Mid- To Late-Season Weed Detection In Uav Imagery, Arun Narenthiran Veeranampalayam Sivakumar, Jiating Li, Stephen Scott, Eric T. Psota, Amit J. Jhala, Joe D. Luck, Yeyin Shi

Department of Agricultural and Biological Systems Engineering: Faculty Publications

Mid- to late-season weeds that escape from the routine early-season weed management threaten agricultural production by creating a large number of seeds for several future growing seasons. Rapid and accurate detection of weed patches in field is the first step of site-specific weed management. In this study, object detection-based convolutional neural network models were trained and evaluated over low-altitude unmanned aerial vehicle (UAV) imagery for mid- to late-season weed detection in soybean fields. The performance of two object detection models, Faster RCNN and the Single Shot Detector (SSD), were evaluated and compared in terms of weed detection performance using mean …

Fast Decision-Making Under Time And Resource Constraints, Kyle Gabriel Lassak

Graduate Theses, Dissertations, and Problem Reports (ETD)

Practical decision makers are inherently limited by computational and memory resources as well as the time available in which to make decisions. To cope with these limitations, humans actively seek methods which limit their resource demands by exploiting structure within the environment and exploiting a coupling between their sensing and actuation to form heuristics for fast decision-making. To date, such behavior has not been replicated in artificial agents. This research explores how heuristics may be incorporated into the decision-making process to quickly make high-quality decisions through the analysis of a prominent case study: the outfielder problem. In the outfielder problem, …

Robot Simulation Analysis, Jacob Miller, Jeremy Evert

Student Research

• Simulate virtual robot for test and analysis

• Analyze SLAM solutions using ROS

• Assemble a functional Turtlebot

• Emphasize projects related to current research trajectories for NASA, and general robotics applications

A Co-Optimal Coverage Path Planning Method For Aerial Scanning Of Complex Structures, Zhexiong Shang, Justin Bradley, Zhigang Shen

Department of Construction Engineering and Management: Faculty Publications

The utilization of unmanned aerial vehicles (UAVs) in survey and inspection of civil infrastructure has been growing rapidly. However, computationally efficient solvers that find optimal flight paths while ensuring high-quality data acquisition of the complete 3D structure remains a difficult problem. Existing solvers typically prioritize efficient flight paths, or coverage, or reducing computational complexity of the algorithm – but these objectives are not co-optimized holistically. In this work we introduce a co-optimal coverage path planning (CCPP) method that simultaneously co-optimizes the UAV path, the quality of the captured images, and reducing computational complexity of the solver all while adhering to …

Exercises Integrating High School Mathematics With Robot Motion Planning, Ronald I. Greenberg, George K. Thiruvathukal

Computer Science: Faculty Publications and Other Works

This paper presents progress in developing exercises for high school students incorporating level-appropriate mathematics into robotics activities. We assume mathematical foundations ranging from algebra to precalculus, whereas most prior work on integrating mathematics into robotics uses only very elementary mathematical reasoning or, at the other extreme, is comprised of technical papers or books using calculus and other advanced mathematics. The exercises suggested are relevant to any differerential-drive robot, which is an appropriate model for many different varieties of educational robots. They guide students towards comparing a variety of natural navigational strategies making use of typical movement primitives. The exercises align …

Optimal Sampling Paths For Autonomous Vehicles In Uncertain Ocean Flows, Andrew J. De Stefan

Dissertations

Despite an extensive history of oceanic observation, researchers have only begun to build a complete picture of oceanic currents. Sparsity of instrumentation has created the need to maximize the information extracted from every source of data in building this picture. Within the last few decades, autonomous vehicles, or AVs, have been employed as tools to aid in this research initiative. Unmanned and self-propelled, AVs are capable of spending weeks, if not months, exploring and monitoring the oceans. However, the quality of data acquired by these vehicles is highly dependent on the paths along which they collect their observational data. The …

Dimensional Analysis Of Robot Software Without Developer Annotations, John-Paul W. Ore

School of Computing: Dissertations, Theses, and Student Research

Robot software risks the hazard of dimensional inconsistencies. These inconsistencies occur when a program incorrectly manipulates values representing real-world quantities. Incorrect manipulation has real-world consequences that range in severity from benign to catastrophic. Previous approaches detect dimensional inconsistencies in programs but require extra developer effort and technical complications. The extra effort involves developers creating type annotations for every variable representing a real-world quantity that has physical units, and the technical complications include toolchain burdens like specialized compilers or type libraries.

To overcome the limitations of previous approaches, this thesis presents novel methods to detect dimensional inconsistencies without developer annotations. We …

Mathematics And Programming Exercises For Educational Robot Navigation, Ronald I. Greenberg

Computer Science: Faculty Publications and Other Works

This paper points students towards ideas they can use towards developing a convenient library for robot navigation, with examples based on Botball primitives, and points educators towards mathematics and programming exercises they can suggest to students, especially advanced high school students.

Chatbots: Conversation Killers Or Makers?, Jing Jiang

MITB Thought Leadership Series

Whether you’re aware of it or not, the chances are you’ve been chatting to robots of late. While these bots are faceless and unseen, don’t be fooled into thinking they aren’t there. In fact, chatbots, have been around since the 1960s at least, but with the progress in artificial intelligence, cloud computing and voice recognition, they’ve received both a functionality and a popularity boost. From the cosmetic to the life-changing, nowadays, chatbots can do anything from helping a person lose weight to assisting refugees applying for asylum.

Labeling Paths With Convolutional Neural Networks, Sean Wallace, Kyle Wuerch

Computer Engineering

With the increasing development of autonomous vehicles, being able to detect driveable paths in arbitrary environments has become a prevalent problem in multiple industries. This project explores a technique which utilizes a discretized output map that is used to color an image based on the confidence that each block is a driveable path. This was done using a generalized convolutional neural network that was trained on a set of 3000 images taken from the perspective of a robot along with matching masks marking which portion of the image was a driveable path. The techniques used allowed for a labeling accuracy …

Confucian Robot Ethics, Qin Zhu, Tom Williams, Ruchen Wen

Computer Ethics - Philosophical Enquiry (CEPE) Proceedings

In the literature of artificial moral agents (AMAs), most work is influenced by either deontological or utilitarian frameworks. It has also been widely acknowledged that these Western “rule-based” ethical theories have encountered both philosophical and computing challenges. To tackle these challenges, this paper explores a non-Western, role-based, Confucian approach to robot ethics. In this paper, we start by providing a short introduction to some theoretical fundamentals of Confucian ethics. Then, we discuss some very preliminary ideas for constructing a Confucian approach to robot ethics. Lastly, we briefly share a couple of empirical studies our research group has recently conducted that …

The Future Robo-Advisor, Catalin Burlacu

MITB Thought Leadership Series

The accelerated digitalisation of both people and business around the world today is having a huge impact on the investment management and advisory space. The addition of new and vastly larger data sets, as well as exponentially more sophisticated analytical tools to turn that data into usable information is constantly changing the way investments are decided on, made and managed.

Autonomous Watercraft Simulation And Programming, Nicholas J. Savino

Student Scholar Showcase

Automation of various modes of transportation is thought to make travel more safe and efficient. Over the past several decades, advances to semi-autonomous and autonomous vehicles have led to advanced autopilot systems on planes and boats, and an increasing popularity of self-driving cars. We predicted the motion of an autonomous vehicle using simulations in Python. The simulation models the motion of a small scale watercraft, which can then be built and programmed using an Arduino Microcontroller. We examined different control methods for a simulated rescue craft to reach a target. We also examined the effects of different factors, such as …

Integrating Mathematics And Educational Robotics: Simple Motion Planning, Ronald I. Greenberg, George K. Thiruvathukal, Sara T. Greenberg

Computer Science: Faculty Publications and Other Works

This paper shows how students can be guided to integrate elementary mathematical analyses with motion planning for typical educational robots. Rather than using calculus as in comprehensive works on motion planning, we show students can achieve interesting results using just simple linear regression tools and trigonometric analyses. Experiments with one robotics platform show that use of these tools can lead to passable navigation through dead reckoning even if students have limited experience with use of sensors, programming, and mathematics.

Robotic Motion Generation By Using Spatial-Temporal Patterns From Human Demonstrations, Yongqiang Huang

USF Tampa Graduate Theses and Dissertations

Robots excel in manufacturing facilities because the tasks are repetitive and do not change. However, when the tasks change, which happens in almost all tasks that humans perform daily, such as cutting, pouring, and grasping, etc., robots perform much worse. We aim at teaching robots to perform tasks that are subject to change using demonstrations collected from humans, a problem referred to as learning from demonstration (LfD).

LfD consists of two parts: the data of human demonstrations, and the algorithm that extracts knowledge from the data to perform the same motions. Similarly, this thesis is divided into two parts. The …

Volumetric Error Compensation For Industrial Robots And Machine Tools, Le Ma

Doctoral Dissertations

“A more efficient and increasingly popular volumetric error compensation method for machine tools is to compute compensation tables in axis space with tool tip volumetric measurements. However, machine tools have high-order geometric errors and some workspace is not reachable by measurement devices, the compensation method suffers a curve-fitting challenge, overfitting measurements in measured space and losing accuracy around and out of the measured space. Paper I presents a novel method that aims to uniformly interpolate and extrapolate the compensation tables throughout the entire workspace. By using a uniform constraint to bound the tool tip error slopes, an optimal model with …

Controlled Switching In Kalman Filtering And Iterative Learning Controls, He Li

Masters Theses

“Switching is not an uncommon phenomenon in practical systems and processes, for examples, power switches opening and closing, transmissions lifting from low gear to high gear, and air planes crossing different layers in air. Switching can be a disaster to a system since frequent switching between two asymptotically stable subsystems may result in unstable dynamics. On the contrary, switching can be a benefit to a system since controlled switching is sometimes imposed by the designers to achieve desired performance. This encourages the study of system dynamics and performance when undesired switching occurs or controlled switching is imposed. In this research, …

Effective Plant Discrimination Based On The Combination Of Local Binary Pattern Operators And Multiclass Support Vector Machine Methods, Vi N T Le, Beniamin Apopei, Kamal Alameh

Research outputs 2014 to 2021

Accurate crop and weed discrimination plays a critical role in addressing the challenges of weed management in agriculture. The use of herbicides is currently the most common approach to weed control. However, herbicide resistant plants have long been recognised as a major concern due to the excessive use of herbicides. Effective weed detection techniques can reduce the cost of weed management and improve crop quality and yield. A computationally efficient and robust plant classification algorithm is developed and applied to the classification of three crops: Brassica napus (canola), Zea mays (maize/corn), and radish. The developed algorithm is based on the …

Exploring Cyber-Physical Systems, Misbah Uddin Mohammed

Graduate Research Theses & Dissertations

The advances in IOT, Computer Vision, AI and Machine Learning have made these technologies ubiquitous to our daily lives. From Smart Phones to Connected Vehicles, Cyber Physical systems have been interspersed into everything we interact in today’s world. The aim or this thesis was to explore these advances in Cyber Physical Systems and analyze the different sectors they were affecting. We then hand-picked certain domains and explored further by carrying out practical projects using some of the latest software and hardware resources available. Technologies like Amazon Alexa services, NVIDIA Jetson boards, TensorFlow, OpenCV, NodeJS were heavily employed in our various …

Collaborative Robotic Path Planning For Industrial Spraying Operations On Complex Geometries, Steven Brown

Graduate Theses and Dissertations

Implementation of automated robotic solutions for complex tasks currently faces a few major hurdles. For instance, lack of effective sensing and task variability – especially in high-mix/low-volume processes – creates too much uncertainty to reliably hard-code a robotic work cell. Current collaborative frameworks generally focus on integrating the sensing required for a physically collaborative implementation. While this paradigm has proven effective for mitigating uncertainty by mixing human cognitive function and fine motor skills with robotic strength and repeatability, there are many instances where physical interaction is impractical but human reasoning and task knowledge is still needed. The proposed framework consists …

Enhancing 3d Visual Odometry With Single-Camera Stereo Omnidirectional Systems, Carlos A. Jaramillo

Dissertations, Theses, and Capstone Projects

We explore low-cost solutions for efficiently improving the 3D pose estimation problem of a single camera moving in an unfamiliar environment. The visual odometry (VO) task -- as it is called when using computer vision to estimate egomotion -- is of particular interest to mobile robots as well as humans with visual impairments. The payload capacity of small robots like micro-aerial vehicles (drones) requires the use of portable perception equipment, which is constrained by size, weight, energy consumption, and processing power. Using a single camera as the passive sensor for the VO task satisfies these requirements, and it motivates the …

Identification And Optimal Linear Tracking Control Of Odu Autonomous Surface Vehicle, Nadeem Khan

Mechanical & Aerospace Engineering Theses & Dissertations

Autonomous surface vehicles (ASVs) are being used for diverse applications of civilian and military importance such as: military reconnaissance, sea patrol, bathymetry, environmental monitoring, and oceanographic research. Currently, these unmanned tasks can accurately be accomplished by ASVs due to recent advancements in computing, sensing, and actuating systems. For this reason, researchers around the world have been taking interest in ASVs for the last decade. Due to the ever-changing surface of water and stochastic disturbances such as wind and tidal currents that greatly affect the path-following ability of ASVs, identification of an accurate model of inherently nonlinear and stochastic ASV system …

A Qualitative Research On Marketing And Sales In The Artificial Intelligence Age, Yin Yang, Keng Siau

Research Collection School Of Computing and Information Systems

The age of artificial intelligence is here! Artificial Intelligence, robotics, machine learning, and automation are impacting the field of marketing and sales in an unprecedented way. In this study, the qualitative research methodology will be used to better understand the revolution and evolution of marketing and sales field in the AI age. Multiple case studies will be performed in various marketing and sales units in different organizations. This research is of value to both academics and practitioners as it aims to provide a detailed analysis and documentation of the changes in marketing and sales functionalities and job markets as AI …

Design And Implementation Of An Artificial Neural Network Controller For Quadrotor Flight In Confined Environment, Ahmed Mekky

Mechanical & Aerospace Engineering Theses & Dissertations

Quadrotors offer practical solutions for many applications, such as emergency rescue, surveillance, military operations, videography and many more. For this reason, they have recently attracted the attention of research and industry. Even though they have been intensively studied, quadrotors still suffer from some challenges that limit their use, such as trajectory measurement, attitude estimation, obstacle avoidance, safety precautions, and land cybersecurity. One major problem is flying in a confined environment, such as closed buildings and tunnels, where the aerodynamics around the quadrotor are affected by close proximity objects, which result in tracking performance deterioration, and sometimes instability. To address this …