Robotics Commons^™

Training Uav Teams With Multi-Agent Reinforcement Learning Towards Fully 3d Autonomous Wildfire Response, Bryce Hopkins

All Theses

As climate-exacerbated wildfires increasingly threaten landscapes and communities, there is an urgent and pressing need for sophisticated fire management technologies. Coordinated teams of Unmanned Aerial Vehicles (UAVs) present a promising solution for detection, assessment, and even incipient-stage suppression – especially when integrated into a multi-layered approach with other recent wildfire management technologies such as geostationary/polar-orbiting satellites and CCTV detection networks. However, there remains significant challenges in developing the necessary sensing, navigation, coordination, and communication subsystems that enable intelligent UAV teams. Further, federal regulations governing UAV deployment and autonomy pose constraints on real-world aerial testing, creating a disconnect between theoretical research …

Convex Approach To Data-Driven Optimal Control With Safety Constraints Using Linear Transfer Operator, Joseph Raphel Moyalan

All Dissertations

This thesis is concerned with the data-driven solution to the optimal control problem with safety constraints for a class of control-affine nonlinear systems. Designing optimal control satisfying safety constraints is a problem of interest in various applications, including robotics, power systems, transportation networks, and manufacturing. This problem is known to be non-convex. One of this thesis's main contributions is providing a convex formulation to this non-convex problem. The second main contribution is providing a data-driven framework for solving the control problem with safety constraints. The linear operator theoretic framework involving Perron-Frobenius and Koopman operators provides the convex formulation and associated …

Authenticated Diagnosing Of Covid-19 Using Deep Learning-Based Ct Image Encryption Approach, Mohamed Attia Abdelgwad, Amira Hassan Abed, Mahmoud Bahloul

Future Computing and Informatics Journal

Researchers are motivated to use artificial intelligence in biometrics, medical imaging encryption, as well as cybersecurity due to its rapid progress. An encryption method for CT scans—which are used to diagnose COVID-19 disease—is proposed in this study. The suggested encryption method creates a connection among an individual's face picture and CT image to increase confidentiality. The simple CT picture is first enhanced with a host image. An encryption key is multiplied by the final result. This key is produced by applying a Convolutional Neural Network (CNN) to recognize characteristics from people's face photographs. Additionally, a straightforward CNN with three convolutional …

Deep Learning For Multiple Unmanned Aerial Vehicle Coordination In Air Corridors, Liangkun Yu

Electrical and Computer Engineering ETDs

In the future, city skies will be filled with Unmanned Aerial Vehicles (UAVs) for rapid urban transport, including parcel deliveries and air taxis. NASA's Urban Air Mobility (UAM) envisions UAVs navigating air corridors. These virtual pathways ensure safety and compliance with regulations. However, current research on UAM practical applications is limited. This dissertation focuses on designing air corridors, developing UAV control systems, and ensuring the robustness of control algorithms against disturbances in real-world environments.

Our design features an air corridor system with horizontal lanes and on-off ramps, conceptualized as cylindrical spaces and tori, respectively. To enable each UAV to locally …

Studying The Performance Of Object Recognition With Fusion Of Visible Light And Infrared Images With Neural Networks, Plamen Petkov

Doctoral Dissertations and Master's Theses

Neural networks have been used for object detection and recognition in both color and intensity camera images. As the use of infrared cameras, colloquially termed thermal cameras, has increased and costs have decreased, object detection and recognition in infrared camera images have been increasingly studied. An infrared image is treated as an intensity image, just like a grayscale camera image, except the intensity corresponds to infrared radiation instead of visible light. The information provided by these two types of images are different, especially in different lighting and environmental situations, and some types of objects are more easily recognized in visible …

Autonomous Apple Harvester Robot, Jack Ryan Cline, Tyus Green, Devon Woolston

Electrical Engineering

As agricultural demands rise and manual labor costs increase, there has become a dire need to automate apple harvesting. However, the precision and speed necessary for cost-efficient apple harvesting pose a significant challenge for robotic automation. To maintain cost-effective production, a harvester must be able to operate fast enough and long enough to compete with human labor. It must also be able to navigate and traverse apple orchards autonomously and pick apples without damaging the fruit or tree. This project presents an apple harvesting robot that uses a Mask R-CNN vision system with an RGB-D camera to detect the location …

Securing The Skies: Safety-Constrained Decentralized Multi-Uav Coordination With Deep Reinforcement Learning, Jean-Elie Pierre

Electrical and Computer Engineering ETDs

In the dynamic landscape of autonomous aerial systems, the integration of uncrewed aerial vehicles (UAVs) has sparked a paradigm shift, offering unprecedented opportunities and challenges in collaborative decision-making and navigation. This thesis explores the application of multi-agent reinforcement learning (MARL) for the planning and coordination of UAVs in complex environments.

The first part of this thesis provides an introduction to single-agent reinforcement learning and MARL. We provide examples of the use of MARL for countering uncrewed aerial systems (C-UAS). We formulate the Counter-UAS problem as a multiagent partially observable Markov decision process (MAPOMDP), and we propose Multi-AGent partial observable deep …

Generalized Model To Enable Zero-Shot Imitation Learning For Versatile Robots, Yongshuai Wu

Master's Theses

The rapid advancement in Deep Learning (DL), especially in Reinforcement Learning (RL) and Imitation Learning (IL), has positioned it as a promising approach for a multitude of autonomous robotic systems. However, the current methodologies are predominantly constrained to singular setups, necessitating substantial data and extensive training periods. Moreover, these methods have exhibited suboptimal performance in tasks requiring long-horizontal maneuvers, such as Radio Frequency Identification (RFID) inventory, where a robot requires thousands of steps to complete.

In this thesis, we address the aforementioned challenges by presenting the Cross-modal Reasoning Model (CMRM), a novel zero-shot Imitation Learning policy, to tackle long-horizontal robotic …

Summonable Construction Delivery Robot, Kevin M. Lewis

Honors Capstones

In many different construction industries, there is a need for tools, parts, and other necessary items to be transported quickly and efficiently over various types of terrain. Human resources have often been used to address these needs, which can become very time and cost inefficient over long periods. The design proposal here is aimed at addressing this need by developing an autonomous outdoor mobile robot based on a quadrupedal robot design. This approach differs by incorporating a wheeled and quadrupedal hybrid actuation system that provides terrain negotiation and speed at the appropriate times. The team uses Robot Operating System (ROS) …

Hand Movement Analysis For Surgical Suturing Skill Assessment, Amir Mehdi Shayan

All Theses

To enhance patient safety, surgical education is increasingly incorporating simulation for formative skills assessment and training. However, many standardized assessment tools rely on human raters for performance assessment, which is resource-intensive and subjective. Simulators that provide automated and objective metrics from sensor data can address this limitation. This thesis presents an instrumented bench suturing simulator, patterned after the Clock Face (CF) radial suturing model from the Fundamentals of Vascular Surgery (FVS), for automated and objective assessment of open suturing skills by particularly focusing on biomechanical analysis of hand movements. For this research, 97 participants (35 attending surgeons and fellows, 32 …

State Omniscience For Cooperative Local Catalog Maintenance Of Close Proximity Satellite Systems, Chris Hays

Doctoral Dissertations and Master's Theses

Resiliency in multi-agent system navigation is reliant on the inherent ability of the system to withstand, overcome, or recover from adverse conditions and disturbances. In large part, resiliency is achieved through reducing the impact of critical failure points to the success and/or performance of the system. In this view, decentralized multi-agent architectures have become an attractive solution for multi-agent navigation, but decentralized architectures place the burden of information acquisition directly on the agents themselves. In fact, the design of distributed estimators has been a growing interest to enable complex multi-sensor/multi-agent tasks. In such scenarios, it is important that each local …

Nonlinear Guidance And Control Of Unmanned Aerial Manipulators For Delivering A Payload On A Moving Platform, Ravi Gyawali

Mechanical and Aerospace Engineering Theses - Archive

Unmanned Aerial manipulators (UAMs) are a class of Unmanned Aerial Vehicles (UAVs) equipped with a manipulator. By combining the aerial mobility of a UAV with a manipulator's dexterity, these hybrid systems can perform a wide range of complex tasks while reducing risks and costs. As a result, they are increasingly being utilized for military, industrial, and agricultural applications.

The thesis presents a novel approach for a multi-UAM system to collaboratively deliver a payload on a stationary or maneuvering platform. A sliding-mode-based guidance law, sourced from existing literature, is integrated with a combined control technique for the UAVs and their respective …

The Integration Of Neuromorphic Computing In Autonomous Robotic Systems, Md Abu Bakr Siddique

Dissertations, Master's Theses and Master's Reports

Deep Neural Networks (DNNs) have come a long way in many cognitive tasks by training on large, labeled datasets. However, this method has problems in places with limited data and energy, like when planetary robots are used or when edge computing is used [1]. In contrast to this data-heavy approach, animals demonstrate an innate ability to learn by communicating with their environment and forming associative memories among events and entities, a process known as associative learning [2-4]. For instance, rats in a T-maze learn to associate different stimuli with outcomes through exploration without needing labeled data [5]. This learning paradigm …

Implementing Associative Learning Using Neuromorphic Robot, Vinay Kumar Pillalamarri

Dissertations, Master's Theses and Master's Reports

Associative learning, a key cognitive process seen across the animal kingdom, enables organisms to form connections between stimuli and adapt their behaviors based on past experiences. A particularly powerful example is fear conditioning, where animals learn to associate a neutral stimulus with an aversive one, allowing them to predict and avoid potential threats. Inspired by this mechanism, this project implements associative learning on an unmanned ground vehicle (UGV) to develop adaptive behavior through neuromorphic principles. Utilizing Nengo for neural modeling, the UGV learns to associate visual (red color) and tactile (vibration) stimuli through Hebbian learning, a biologically inspired synaptic adaptation …

Autonomous Damage And Structure Scanning Drone, Natasha Ninan, Amber Long, Lee Nestor, Emmanuel Jensen

Williams Honors College, Honors Research Projects

Remote damage analysis plays a crucial role in lowering the risk associated with human presence at dangerous sites. This project focuses on developing a system for remote structural examination using photogrammetric techniques and analysis. The system utilizes a drone equipped with cameras for photogrammetry, and LiDARs for navigation. This setup can enable efficient structural model generation with the collection of images from multiple points. A novel structural analysis is performed to detect potential damage or points of failure in the structure.

Key components include a teleoperated drone, a software pipeline for photogrammetric analysis, collision protection mechanisms, and a user interface. …

Autonomous Basketball Court Creation Robot, Bryce Haldeman, Tyler Gray, Dalon Vura

Williams Honors College, Honors Research Projects

The Autonomous Basketball Court Outlining System presents a comprehensive solution for precision court marking. Powered by a 24V lithium-ion battery and driven by a single ST microcontroller, the system autonomously marks the outline of a half basketball court using predefined algorithms. User-friendly features include easy loading of marking material, actuated by gravity or a small servo motor depending on material of choice, ensuring intuitive operation. Safety is prioritized, with the servo motor eliminating high-pressure concerns, and the system maintains a controlled speed accounting for user well-being. Two step and direction servo motors enable accurate linear displacement, facilitating straight lines, and …

Efficient Connectivity Management And Path Planning For Iot And Uav Networks, Amirahmad Chapnevis

Theses and Dissertations

This dissertation explores how to better manage resources in mobile networks, especially for enhancing the performance of Unmanned Aerial Vehicles (UAV)-supported IoT networks. We explored ways to set up a flexible communication architecture that can handle large IoT deployments by making good use of mobile core network resources like bearers and data paths. We developed strategies that meet the needs of IoT networks and enhance network performance. We also developed and tested a system that combines traffic from several mobile devices that use the same user identity and network resources within the core mobile network. We used everyday smartphones, SIM …

Observability-Aware Path Planning For Autonomous Navigation, Raymond Brink Neistat

Open Access Master's Theses

Terrain-Aided Navigation (TAN) is a popular method of localization for GPS-denied vehicles, particularly in the marine domain. There are many ways to perform TAN in a marine setting, such as Bathymetric SLAM (BSLAM) and Bayesian Filtering with the aid of an a priori map. These techniques have been studied extensively, but show an overall lack of rigorous observability analyses. Without nonlinear observability analyses, TAN practitioners do not have an analytical indicator to know which areas of terrain will provide opportunities for the best localization performance. This thesis reviews current developments in the endeavors of nonlinear observability analyses as well as …

On Uncertainty For Ill-Posed Robot Decision Problems, Jared Joseph Beard

Graduate Theses, Dissertations, and Problem Reports (ETD)

As robots adopt more real world responsibilities, they will be expected to solve more complicated problems. In some cases limited prior knowledge will result in unmodelled environmental conditions; in others, multiple users may have competing perspectives on how to frame a decision problem. Many existing frameworks, namely Markov decision processes (MDP) presuppose users have identified a specific problem with models sufficient to solve or learn a problem. If we wish to extend MDPs to novel problems or those heavily dependent on user feedback, autonomous decision makers must be able to identify limitations in how a given problem is framed and …

External Runtime Execution Monitoring Of A Cyber Physical System Via Trace Interfaces, Peter Vaughan Truslow

Theses and Dissertations

In the past two decades, Unmanned Aerial Systems have progressed from expensive military hardware or one-off custom builds, to include off-the-shelf drones that can be purchased for a rather affordable price and flown by nearly anyone. As the technology and performance have improved, the door is opened to applications that require operation in environments where the consequences for failure are high, such as operating in the navigable airspace or in urban environments, or with human passengers. This requires a great deal of trust in the reliability and integrity of the control systems of the aircraft. A method of monitoring the …

Enhancing Robotic Exploration Through Semantically-Guided Sampling Strategies, Christopher Alexander Arend Tatsch

Graduate Theses, Dissertations, and Problem Reports (ETD)

From space and deep-sea exploration to disaster response and environmental monitoring, autonomous robots are essential for advancing science, improving safety, and addressing critical challenges. This dissertation introduces a novel open-source strategy for autonomous robotic exploration: the Semantically-Guided Exploration (SGE) framework. Designed for ground vehicles, SGE integrates semantic understanding into the autonomous exploration process, improving decision-making in complex environments. Specifically, the proposed sampling-based approach uses the information from the semantic segmentation of RGB images and depth images to guide the robot's selection of exploration goals. This method enables the robot to steer away from potential dangers such as large rocks and …

Control Of Fully-Actuated Aerial Manipulators And Omni-Directional Multirotors, Riley M. Mccarthy

Mechanical Engineering ETDs

This thesis details the system modeling, design, control, simulation, construction, and
testing of both a fully-actuated and omni-directional multirotor aerial system created
for the primary purpose of performing active tasks with their environment. This work
verifies the capabilities of both systems through empirical testing, and demonstrates
how through the use of new control methods and physical designs multirotors can
expand their purpose from passive inspection based tasks to active contact based
tasks. These systems take advantage of newly implemented control allocation features present in the PX4 flight control software, version 1.14. The use of which makes designing controllers for such …

Safety-Aware Autonomous Robot Navigation, Mapping And Control By Optimization Techniques, Tingjun Lei

Theses and Dissertations

The realm of autonomous robotics has seen impressive advancements in recent years, with robots taking on essential roles in various sectors, including disaster response, environmental monitoring, agriculture, and healthcare. As these highly intelligent machines continue to integrate into our daily lives, the pressing imperative is to elevate and refine their performance, enabling them to adeptly manage complex tasks with remarkable efficiency, adaptability, and keen decision-making abilities, all while prioritizing safety-aware navigation, mapping, and control systems. Ensuring the safety-awareness of these robotic systems is of paramount importance in their development and deployment. In this research, bio-inspired neural networks, nature-inspired intelligence, deep …

Implementation Of Adas And Autonomy On Unlv Campus, Zillur Rahman

UNLV Theses, Dissertations, Professional Papers, and Capstones

The integration of Advanced Driving Assistance Systems (ADAS) and autonomous driving functionalities into contemporary vehicles has notably surged, driven by the remarkable progress in artificial intelligence (AI). These AI systems, capable of learning from real-world data, now exhibit the capability to perceive their surroundings via a suite of sensors, create optimal routes from source to destination, and execute vehicle control akin to a human driver.

Within the context of this thesis, we undertake a comprehensive exploration of three distinct yet interrelated ADAS and Autonomy projects. Our central objective is the implementation of autonomous driving(AD) technology at UNLV campus, culminating in …

A Modular Framework For Surface-Embedded Actuation And Optical Sensing In Soft Robots., Paul Bupe Jr

Electronic Theses and Dissertations

This dissertation explores the development and integration of modular technologies in soft robotics, with a focus on the OptiGap sensor system. OptiGap serves as a simple, flexible, cost-effective solution for real-time sensing of bending and deformation, validated through simulation and experimentation. Working as part of an emerging category of soft robotics called Soft, Curved, Reconfigurable, Anisotropic Mechanisms, or SCRAMs, this research also introduces the Thermally-Activated SCRAM Limb (TASL) technology, which employs shape-memory alloy (SMA) wire embedded in curved sheets for surface actuation and served as the initial inspiration for OptiGap. In addition, the EneGate system is presented as a complementary …

Accelerating Machine Learning Inference For Satellite Component Feature Extraction Using Fpgas., Andrew Ekblad

Theses and Dissertations

Running computer vision algorithms requires complex devices with lots of computing power, these types of devices are not well suited for space deployment. The harsh radiation environment and limited power budgets have hindered the ability of running advanced computer vision algorithms in space. This problem makes running an on-orbit servicing detection algorithm very difficult. This work proposes using a low powered FPGA to accelerate the computer vision algorithms that enable satellite component feature extraction. This work uses AMD/Xilinx’s Zynq SoC and DPU IP to run model inference. Experiments in this work centered around improving model post processing by creating implementations …

Modeling And Compensating Of Noise In Time-Of-Flight Sensors, Bryan Rodriguez

Electrical Engineering Theses and Dissertations

Three-dimensional (3D) sensors provide the ability to perform contactless measurements of objects and distances that are within their field of view. Unlike traditional two-dimensional (2D) cameras, which only provide RGB data about objects within a scene, 3D sensors are able to directly provide depth information for objects within a scene. Of these 3D sensing technologies, Time-of-Flight (ToF) sensors are becoming more compact which allows them to be more easily integrated with other devices and to find use in more applications. ToF sensors also provide several benefits over other 3D sensing technologies that increase the types of applications where ToF sensors …

A Study On Rapidly Exploring Random Tree Algorithms For Robot Path Planning, Sahil Sharma

Master's Theses

Robot path planning is a critical feature of autonomous systems. Rapidly-exploring Random Trees (RRT) is a path planning technique that randomly samples the robot configuration space to find a path between the start and end point. This thesis studies and compares the performance of four important RRT algorithms, namely, the original RRT, the optimal RRT (also termed RRT*), RRT*-Smart, and Informed RRT* for six different environments. The performance measures include the final path length (which is also the shortest path length found by each algorithm), time to find the first path, run time (of 1000 iterations) for each algorithm, total …

Faults Segmentation In Levee Systems Using Deep Learning Approaches, Manisha Panta

LSU New Orleans Theses and Dissertations

Levees are earthen structures constructed to mitigate flooding in low-lying areas. Although levee systems can reduce flood risks, they cannot completely eliminate them. Failures within flood control systems due to inadequate maintenance or strong water currents can lead to significant property damage and catastrophic loss of life, as was seen during Hurricane Katrina. Consequently, regular inspections are essential to identify and address any issues with the levees promptly. However, current inspection methods rely on manual techniques that are time-consuming, labor-intensive, and prone to human error. Therefore, this study proposes using deep learning models for more efficient and frequent assessment of …

Accurate Orientation Control Of Tendon Driven Continuum Robots That Exhibit Elasticity, Manu Srivastava

All Dissertations

This dissertation makes new contributions to the modeling and implementation of Tendon Driven Continuum Robots (TDCRs). Specifically, motivated by 3D printing of concrete using a continuum hose robot in construction applications, we focus on TDCRs featuring compliance in the robot backbone and actuating tendons, e.g. surgical robots/endoscopes/catheters with tendon actuation. We expand previous mechanics-based models to show how and why such compliance significantly restricts performance when traditional kinematics-based planning and control techniques are applied.

The main contribution of this work is a new Elasticity Compensation(EC) model that explains why the ad hoc approach of preloading/pretensioning the tendons compensates for compliance …