Robotics Commons^™

Leveraging Area Bounds Information For Autonomous Multi-Robot Exploration, Tsung-Ming Liu, Damian M. Lyons

Faculty Publications

In this paper we propose an approach, the Space-Based Potential Field (SBPF) approach, to controlling multiple robots for area exploration missions that focus on robot dispersion. The SBPF method is based on a potential field approach that leverages knowledge of the overall bounds of the area to be explored. This additional information allows a simpler potential field control strategy for all robots but which nonetheless has good dispersion and overlap performance in all the multi-robot scenarios while avoiding potential minima. Both simulation and robot experimental results are presented as evidence.

Getting It Right The First Time: Verification Of Autonomous Behavior-Based Multirobot Missions, Ronald C. Arkin, Damian M. Lyons

Faculty Publications

No abstract provided.

Eliminating Mutual Views In Fusion Of Ranging And Rgb-D Data From Robot Teams Operating In Confined Areas, Damian M. Lyons, Karma Shrestha

Faculty Publications

We address the problem of fusing laser and RGB-Data from multiple robots operating in close proximity to one another. By having a team of robots working together, a large area can be scanned quickly, or a smaller area scanned in greater detail. However, a key aspect of this problem is the elimination of the spurious readings due to the robots operating in close proximity. While there is an extensive literature on the mapping and localization aspect of this problem, our problem differs from the dynamic map problem in that it involves at one kind of transient map feature, robots viewing …

Mobile Sensor Data Mining, Jeff Lockhart Fcrh '13

The Fordham Undergraduate Research Journal

At an ever increasing rate, the smartphones and other devices people carry with them in their everyday lives are packed with sensors and processing power. This provides an unprecedented opportunity to apply data mining techniques to people’s activities as they go about their daily lives, without changing their routine. The goal of the Wireless Sensor Data Mining (WISDM) Project is to explore the possibilities of data mining on these powerful mobile platforms.1 Data mining involves extracting knowledge from data using computer algorithms. A major sensor in these devices is the tri-axial accelerometer originally included for screen rotation and advanced gaming. …

Getting It Right The First Time: Robot Mission Guarantees In The Presence Of Uncertainty, Damian M. Lyons, Ronald C. Arkin, Paramesh Nirmal, Shu Jiang, Tsung-Ming Liu, J. Deeb

Faculty Publications

Certain robot missions need to perform predictably in a physical environment that may only be poorly characterized in advance. We have previously developed an approach to establishing performance guarantees for behaviorbased controllers in a process-algebra framework. We extend that work here to include random variables, and we show how our prior results can be used to generate a Dynamic Bayesian Network for the coupled system of program and environment model. Verification is reduced to a filtering problem for this network. Finally, we present validation results that demonstrate the effectiveness of the verification of a multiple waypoint robot mission using this …

Getting It Right The First Time: Robot Mission Guarantees In The Presence Of Uncertainty, Damian Lyons, Ron Arkin, Paramesh Nirmal, Shu Jiang, Tsung-Ming Liu, Julia Deeb

Faculty Publications

Abstract—Certain robot missions need to perform predictably in a physical environment that may only be poorly characterized in advance. We have previously developed an approach to establishing performance guarantees for behavior-based controllers in a process-algebra framework. We extend that work here to include random variables, and we show how our prior results can be used to generate a Dynamic Bayesian Network for the coupled system of program and environment model. Verification is reduced to a filtering problem for this network. Finally, we present validation results that demonstrate the effectiveness of the verification of a multiple waypoint robot mission using this …

Performance Guarantees For C-Wmd Robot Missions, Shu Jiang, Ronald C. Arkin, Damian M. Lyons, Tsung-Ming Liu, Dagon Harrington

Faculty Publications

Robotics has been considered as one of the five key technology areas for defense against attacks with weapons of mass destruction (WMD). However, due to the mass impact nature of WMD, failures of counter-WMD (C-WMD) missions can have catastrophic consequences.To ensure robot's success in carrying out C-WMD missions, we have developed a novel verification framework in providing performance guarantees for behavior-based and probabilistic robot algorithms in complex real-world environments. This paper describes the system architecture and discusses how the verification framework can be used to provide pre-mission performance guarantees for robots in executing C-WMD missions.

Verifying Performance For Autonomous Robot Missions With Uncertainty, Damian M. Lyons, Ronald Arkin, Tsung-Ming Liu, Shu Jiang, Paramesh Nirmal

Faculty Publications

Establishing performance guarantees for robot missions is especially important for C-WMD applications. Software verification techniques, such as model checking (Clark 1999, Jhala & Majumdar 2009), can be applied to robotic applications but characteristics of this application area, including addition of a robot environment model and handling continuous spatial location well, exacerbate state explosion, a key weakness of these methods. We have proposed an approach to verifying robot missions that shifts the focus from state-based analysis onto the solution of a set of flow equations (Lyons et al. 2012). The key novelty introduced in this paper is a probabilistic spatial representation …

Performance Verification For Behavior-Based Robot Missions, Damian M. Lyons, Ronald Arkin, Shu Jiang, Tsung-Ming Liu, Paramesh Nirmal, J. Deeb

Faculty Publications

Certain robot missions need to perform predictably in a physical en-vironment that may only be poorly characterized in advance. This requirement raises many issues for existing approaches to software verification. An approach based on behavior-based controllers in a process-algebra framework is proposed by Lyons et al [15] to side-step state combinatorics. In this paper we show that this approach can be used to generate a Dynamic Bayesian Network for the problem, and that verification is reduced to a filtering problem for this network. We present validation results for the verification of a multiple waypoint robot mission using this approach.

Fusion Of Ranging Data From Robot Teams Operating In Confined Areas, Damian M. Lyons, Karma Shrestha, Tsung-Ming Liu

Faculty Publications

We address the problem of fusing laser ranging data from multiple mobile robots that are surveying an area as part of a robot search and rescue or area surveillance mission. We are specifically interested in the case where members of the robot team are working in close proximity to each other. The advantage of this teamwork is that it greatly speeds up the surveying process; the area can be quickly covered even when the robots use a random motion exploration approach. However, the disadvantage of the close proximity is that it is possible, and even likely, that the laser ranging …

A Cognitive Approach To Vision For A Mobile Robot, Paul Benjamin, Christopher Funk, Damian M. Lyons

Faculty Publications

We describe a cognitive vision system for a mobile robot. This system works in a manner similar to the human vision system, using saccadic, vergence and pursuit movements to extract information from visual input. At each fixation, the system builds a 3D model of a small region, combining information about distance, shape, texture and motion. These 3D models are embedded within an overall 3D model of the robot's environment. This approach turns the computer vision problem into a search problem, with the goal of constructing a physically realistic model of the entire environment. At each step, the vision system selects …

A Software Tool For The Design Of Critical Robot Missions With Performance Guarantees, Damian M. Lyons, Ronald C. Arkin, Paramesh Nirmal, Shu Jiang, Tsung-Ming Liu

Faculty Publications

Deploying a robot as part of a counter-weapons of mass destruction mission demands that the robotic software operates with high assurance. A unique feature of robotic software development is the need to perform predictably in a physical environment that may only be poorly characterized in advance. In this paper, we present an approach to building high assurance software for robot missions carried out in uncertain environments. The software development framework and the verification algorithm, VIPARS, are described in detail. Results are presented for missions including motion and sensing uncertainty, interaction with obstacles, and the use of sensors to guide behavior.

Visual Homing With A Pan-Tilt Based Stereo Camera, Paramesh Nirmal, Damian M. Lyons

Faculty Publications

Visual homing is a navigation method based on comparing a stored image of the goal location and the current image (current view) to determine how to navigate to the goal location. It is theorized that insects, such as ants and bees, employ visual homing methods to return to their nest [1]. Visual homing has been applied to autonomous robot platforms using two main approaches: holistic and feature-based. Both methods aim at determining distance and direction to the goal location. Navigational algorithms using Scale Invariant Feature Transforms (SIFT) have gained great popularity in the recent years due to the robustness of …

Designing Autonomous Robot Missions With Performance Guarantees, Damian M. Lyons, Ronald Arkin, Prem Nirmal, Shu Jiang

Faculty Publications

This paper describes the need and methods required to construct an integrated software verification and mission specification system for use in robotic missions intended for counter-weapons of mass destruction (c-WMD) operations, as part of a 3-year effort for the Defense Threat Reduction Agency. The overall system architecture is described. The principal tool for verification is a process algebra, PARS, based on port automata theory. PARS is introduced, emphasizing its ability to represent probabilistic programs and uncertain and dynamic environments, followed by the analysis of mission properties for an example robotic mission.

Getting It Right The First Time: Predicted Performance Guarantees From The Analysis Of Emergent Behavior In Autonomous And Semi-Autonomous Systems, Ronald C. Arkin, Damian M. Lyons, Shu Jiang, Prem Nirmal, Munzir Zafar

Faculty Publications

A crucially important aspect for mission-critical robotic operations is ensuring as best as possible that an autonomous system be able to complete its task. In a project for the Defense Threat Reduction Agency (DTRA) we are developing methods to provide such guidance, specifically for counter-Weapons of Mass Destruction (C-WMD) missions. In this paper, we describe the scenarios under consideration, the performance measures and metrics being developed, and an outline of the mechanisms for providing performance guarantees.

Navigation Of Uncertain Terrain By Fusion Of Information From Real And Synthetic Imagery, Damian M. Lyons, Prem Nirmal, D. Paul Benjamin

Faculty Publications

We consider the scenario where an autonomous platform that is searching or traversing a building may observe unstable masonry or may need to travel over unstable rubble. A purely behaviour-based system may handle these challenges but produce behaviour that works against long-terms goals such as reaching a victim as quickly as possible. We extend our work on ADAPT, a cognitive robotics architecture that incorporates 3D simulation and image fusion, to allow the robot to predict the behaviour of physical phenomena, such as falling masonry, and take actions consonant with long-term goals.

We experimentally evaluate a cognitive only and reactive only …

Using A Virtual World For Robot Planning, D. Paul Benjamin, John V. Monaco, Yixia Lin, Christopher Funk, Damian M. Lyons

Faculty Publications

We are building a robot cognitive architecture that constructs a real-time virtual copy of itself and its environment, including people, and uses the model to process perceptual information and to plan its movements. This paper describes the structure of this architecture. The software components of this architecture include PhysX for the virtual world, OpenCV and the Point Cloud Library for visual processing, and the Soar cognitive architecture that controls the perceptual processing and task planning. The RS (Robot Schemas) language is implemented in Soar, providing the ability to reason about concurrency and time. This Soar/RS component controls visual processing, deciding …

An Approach To Stereo-Point Cloud Registration Using Image Homographies, Damian M. Lyons, Stephen D. Fox

Faculty Publications

No abstract provided.

Characterizing Performance Guarantees For Real-Time Multiagent Systems Operating In Noisy And Uncertain Environments, Damian M. Lyons, Ronald Arkin, Stephen Fox, Shu Jiang, Prem Nirmal, Munzir Zafar

Faculty Publications

Autonomous robots offer the potential to conduct Counter-Weapons of Mass Destruction (C-WMD) missions in an efficient and robust manner. However, to leverage this potential, a mission designer needs to be able to determine how well a robot system will operate in the noisy and uncertain environments that a C-WMD mission may require. We are developing a software framework for verification of performance guarantees for C-WMD missions based on the MissionLab software system and a novel process algebra approach to representing robot programs and operating environments.

In this paper, we report on our initial research for the Defense Threat Reduction Agency …

A Relaxed Fusion Of Information From Real And Synthetic Images To Predict Complex Behavior, Damian M. Lyons, D. Paul Benjamin

Faculty Publications

An important component of cognitive robotics is the ability to mentally simulate physical processes and to compare the expected results with the information reported by a robot's sensors. In previous work, we have proposed an approach that integrates a 3D game-engine simulation into the robot control architecture. A key part of that architecture is the Match-Mediated Difference (MMD) operation, an approach to fusing sensory data and synthetic predictions at the image level. The MMD operation insists that simulated and predicted scenes are similar in terms of the appearance of the objects in the scene. This is an overly restrictive constraint …

Selection And Recognition Of Landmarks Using Terrain Spatiograms, Damian M. Lyons

Faculty Publications

A team of robots working to explore and map an area may need to share information about landmarks so as to register their local maps and to plan effective exploration strategies. In previous papers we have introduced a combined image and spatial representation for landmarks: terrain spatiograms. We have shown that for manually selected views, terrain spatiograms provide an effective, shared representation that allows for occlusion filtering and a combination of multiple views.

In this paper, we present a landmark saliency architecture (LSA) for automatically selecting candidate landmarks. Using a dataset of 21 outdoor stereo images generated by LSA, we …

Detection And Filtering Of Landmark Occlusions Using Terrain Spatiograms, Damian M. Lyons

Faculty Publications

A team of robots cooperating to quickly produce a map needs to share landmark information between members so that the local maps can be accurately merged. However, the appearance of landmarks as seen by members of the team can change dramatically due to the phenomenon of occlusion.

We have previously presented an approach to landmark representation using Terrain Spatiograms – an extension to image spatiograms in which the spatial information relates to the scene rather than the image. Because this representation preserves depth structure, it is possible to identify and filter potential occlusions.

We present an approach to identifying and …

A Visual Imagination Approach To Cognitive Robotics, Damian M. Lyons, Sirhan Chaudhry, D. Paul Benjamin

Faculty Publications

No abstract provided.

Integrating Perception And Problem Solving To Predict Complex Object Behaviors, Damian M. Lyons, Sirhan Chaudhry, Marius Agica, John Vincent Monaco

Faculty Publications

One of the objectives of Cognitive Robotics is to construct robot systems that can be directed to achieve realworld goals by high-level directions rather than complex, low-level robot programming. Such a system must have the ability to represent, problem-solve and learn about its environment as well as communicate with other agents. In previous work, we have proposed ADAPT, a Cognitive Architecture that views perception as top-down and goaloriented and part of the problem solving process.

Our approach is linked to a SOAR-based problem-solving and learning framework. In this paper, we present an architecture for the perceptive and world modelling components …

Detection And Filtering Of Landmark Occlusions Using Terrain Spatiograms, Damian M. Lyons

Posters

A team of robots cooperating to quickly produce a map needs to share landmark information between team members so that the local maps can be accurately merged. However, a landmark visible to one robot may be partially occluded to another! Terrain Spatiograms are a landmark representation in which the image spatial information relates to the scene rather than the image. This makes it possible to identify and filter potential landmark occlusions. We present an approach to identifying and filtering occlusions using Terrain Spatiograms, and we report experimental results on 20 landmark datasets for varying states of occlusion. We show that …

Synchronizing Real And Predicted Synthetic Video Imagery For Localization Of A Robot To A 3d Environment, Damian M. Lyons, Sirhan Chaudhry, D. Paul Benjamin

Faculty Publications

A mobile robot moving in an environment in which there are other moving objects and active agents, some of which may represent threats and some of which may represent collaborators, needs to be able to reason about the potential future behaviors of those objects and agents. In previous work, we presented an approach to tracking targets with complex behavior, leveraging a 3D simulation engine to generate predicted imagery and comparing that against real imagery. We introduced an approach to compare real and simulated imagery using an affine image transformation that maps the real scene to the synthetic scene in a …

Sharing Landmark Information Using Mixture Of Gaussian Terrain Spatiograms, Damian M. Lyons

Faculty Publications

In this paper we evaluate the use of a novel spatial histogram called the terrain spatiogram as a common representation for exchanging landmark information between robots working as a team to map an area. Individual robots use range sensors to provide the spatial dimension of the spatiogram and video for the image dimension. We have previously shown that terrain spatiograms can be shared between robots in a heterogeneous team to recognize landmarks and to fuse observations from multiple sensors or multiple platforms. A terrain spatiogram using a mixture of Gaussians (MOG) model is introduced and a corresponding normalized spatiogram similarity …

Combining Multiple Scoring Systems For Target Tracking Using Rank-Score Characteristics, Damian M. Lyons, D. Frank Hsu

Faculty Publications

Video target tracking is the process of estimating the current state, and predicting the future state of a target from a sequence of video sensor measurements. Multitarget video tracking is complicated by the fact that targets can occlude one another, affecting video feature measurements in a highly non-linear and difficult to model fashion. In this paper, we apply a multisensory fusion approach to the problem of multitarget video tracking with occlusion. The approach is based on a data-driven method (CFA) to selecting the features and fusion operations that improve a performance criterion. Each sensory cue is treated as a scoring …

Sharing And Fusing Landmark Information In A Team Of Autonomous Robots, Damian M. Lyons

Faculty Publications

A team of robots working to explore and map a space may need to share information about landmarks so as register local maps and to plan effective exploration strategies. In this paper we investigate the use of spatial histograms (spatiograms) as a common representation for exchanging landmark information between robots. Each robot can use sonar, stereo, laser and image information to identify potential landmarks. The sonar, laser and stereo information provide the spatial dimension of the spatiogram in a landmark-centered coordinate frame while video provides the image information. We call the result a terrain spatiogram. This representation can be shared …

Locating And Tracking Objects By Efficient Comparison Of Real And Predicted Synthetic Video Imagery, Damian M. Lyons, D. Paul Benjamin

Faculty Publications

A mobile robot moving in an environment in which there are other moving objects and active agents, some of which may represent threats and some of which may represent collaborators, needs to be able to reason about the potential future behaviors of those objects and agents. In this paper we present an approach to tracking targets with complex behavior, leveraging a 3D simulation engine to generate predicted imagery and comparing that against real imagery. We introduce an approach to compare real and simulated imagery and present results using this approach to locate and track objects with complex behaviors. In this …