Robotics Commons^™

Design And Parametric Study Of A Mems-Based Reservoir Computer For Reinforcement Learning, Andrew P. Carr

Master's Theses

Single-node reservoir computing (RC) is a hardware-efficient approach to machine learning, leveraging the dynamics of physical systems. In this work, two reinforcement learning algorithms, Q-learning and Proximal Policy Optimization (PPO), are applied to a simulated micro-electro-mechanical system (MEMS)-based reservoir computer to solve both discrete and continuous control tasks. MEMS-based reservoirs are low-power, compact, and their natural frequencies (kHz to MHz) pair well with real-time control loops. To explore the relationship between reservoir dynamics and learning performance, a parametric study is conducted on two reservoir hyperparameters, reservoir size and neuron separation, using CartPole-v1 and MountainCar-v0. The RC successfully learns multiple tasks …

An Advanced Hexacopter For Autonomous Exploration Of Mars: Attitude Control And Navigation Strategies, Laura Sopegno

Electronic Theses and Dissertations

Mars exploration has recently witnessed major interest within the scientific community. Unmanned robotic platforms offer reliable solutions to acquire and collect data and information from the Red Planet. Particularly, rovers, landers, and orbiters have significantly shaped planetary exploration on the Moon and Mars, contributing significantly to past missions while also highlighting limitations in their capacity to cover diverse terrains over wide ranges. Given current advances in Unmanned Aircraft Systems (UASs), Unmanned Aerial Vehicles (UAVs) offer promising alternatives for future scientific missions.

It is argued that hexacopters, with their relatively compact design and redundancy, present a promising …

Controlling A Mobile Inverted Pendulum And Optimizing Leaning Angle To Apply Force Using Reinforcement Learning, Aryan Mediratta

2025 Spring Honors Capstone Projects - Archive

Reinforcement Learning is a Machine Learning paradigm that involves simulating learning through rewards and penalties in intelligent systems. This technique is often employed in robotics when traditional control methods are insufficient or when human intuition does not provide a good solution on how to control robot systems, This project involves training a Segway-style Mobile Inverted Pendulum (MIP) robot to balance and push a box forward. The BeagleBone Blue board is used that includes a built-in Inertial Measurement Unit (IMU) and encoder ports. These sensors enable the system to measure its current state. The goal is to find the optimal leaning …

Improving Robustness Of Learning-Based Approaches In Autonomous Systems And Engineering Education, Godwyll Aikins

Theses and Dissertations

This dissertation advances the development of robust learning-based approaches across two complementary domains: engineering education and autonomous systems. Through four studies, this research addresses critical challenges in preparing data-proficient engineers and developing reliable autonomous systems that can operate under uncertainty and incomplete information. The engineering education study examines how mechanical and aerospace engineering undergraduates conceptualize and develop data proficiency skills essential for modern engineering practice. Through interviews with 27 students, the research employs the How People Learn framework to analyze student perspectives on information literacy, data interpretation, and computational thinking. The findings inform pedagogical strategies for developing data proficiency in …

Model-Based Navigation And Control Of Multirotor Uavs: A Machine Learning Approach, Serhat Sönmez

Electronic Theses and Dissertations

In recent decades, unmanned systems, particularly Unmanned Aerial Vehicles (UAVs), have seen significant advancement and unprecedented growth in military, civilian and public domain applications. Scientists have focused on enhancing UAV navigation and control through cutting-edge technologies and support tools. UAVs find applications in many fields, except military, such as agriculture, infrastructure inspection, wildlife monitoring, search and rescue, emergency response, border protection, to name but a few relevant civilian applications. Given the faster-than-exponential increase of available computational power, learning-based algorithms have emerged as a prominent tool for (real-time) multirotor UAV navigation and control. This dissertation centers around the fusion of conventional …

Securing The Skies: Safety-Constrained Decentralized Multi-Uav Coordination With Deep Reinforcement Learning, Jean-Elie Pierre

Electrical and Computer Engineering ETDs

In the dynamic landscape of autonomous aerial systems, the integration of uncrewed aerial vehicles (UAVs) has sparked a paradigm shift, offering unprecedented opportunities and challenges in collaborative decision-making and navigation. This thesis explores the application of multi-agent reinforcement learning (MARL) for the planning and coordination of UAVs in complex environments.

The first part of this thesis provides an introduction to single-agent reinforcement learning and MARL. We provide examples of the use of MARL for countering uncrewed aerial systems (C-UAS). We formulate the Counter-UAS problem as a multiagent partially observable Markov decision process (MAPOMDP), and we propose Multi-AGent partial observable deep …

A Memory Efficient Deep Recurrent Q-Learning Approach For Autonomous Wildfire Surveillance, Jeremy A. Cantor

UNF Graduate Theses and Dissertations

Previous literature demonstrates that autonomous UAVs (unmanned aerial vehicles) have the po- tential to be utilized for wildfire surveillance. This advanced technology empowers firefighters by providing them with critical information, thereby facilitating more informed decision-making processes. This thesis applies deep Q-learning techniques to the problem of control policy design under the objective that the UAVs collectively identify the maximum number of locations that are under fire, assuming the UAVs can share their observations. The prohibitively large state space underlying the control policy motivates a neural network approximation, but prior work used only convolutional layers to extract spatial fire information from …

Multi-Agent Deep Reinforcement Learning For Radiation Localization, Benjamin Scott Totten

Dissertations and Theses

For the safety of both equipment and human life, it is important to identify the location of orphaned radioactive material as quickly and accurately as possible. There are many factors that make radiation localization a challenging task, such as low gamma radiation signal strength and the need to search in unknown environments without prior information. The inverse-square relationship between the intensity of radiation and the source location, the probabilistic nature of nuclear decay and gamma ray detection, and the pervasive presence of naturally occurring environmental radiation complicates localization tasks. The presence of obstructions in complex environments can further attenuate the …

Outdoor Operations Of Multiple Quadrotors In Windy Environment, Deepan Lobo

Dissertations

Coordinated multiple small unmanned aerial vehicles (sUAVs) offer several advantages over a single sUAV platform. These advantages include improved task efficiency, reduced task completion time, improved fault tolerance, and higher task flexibility. However, their deployment in an outdoor environment is challenging due to the presence of wind gusts. The coordinated motion of a multi-sUAV system in the presence of wind disturbances is a challenging problem when considering collision avoidance (safety), scalability, and communication connectivity. Performing wind-agnostic motion planning for sUAVs may produce a sizeable cross-track error if the wind on the planned route leads to actuator saturation. In a multi-sUAV …

Developing Reactive Distributed Aerial Robotics Platforms For Real-Time Contaminant Mapping, Joshua Ashley

Theses and Dissertations--Electrical and Computer Engineering

The focus of this research is to design a sensor data aggregation system and centralized sensor-driven trajectory planning algorithm for fixed-wing aircraft to optimally assist atmospheric simulators in mapping the local environment in real-time. The proposed application of this work is to be used in the event of a hazardous contaminant leak into the atmosphere as a fleet of sensing unmanned aerial vehicles (UAVs) could provide valuable information for evacuation measures. The data aggregation system was designed using a state-of-the-art networking protocol and radio with DigiMesh and a process/data management system in the ROS2 DDS. This system was tested to …

On The Impact Of Gravity Compensation On Reinforcement Learning In Goal-Reaching Tasks For Robotic Manipulators, Jonathan Fugal, Hasan A. Poonawala, Jihye Bae

Electrical and Computer Engineering Faculty Publications

Advances in machine learning technologies in recent years have facilitated developments in autonomous robotic systems. Designing these autonomous systems typically requires manually specified models of the robotic system and world when using classical control-based strategies, or time consuming and computationally expensive data-driven training when using learning-based strategies. Combination of classical control and learning-based strategies may mitigate both requirements. However, the performance of the combined control system is not obvious given that there are two separate controllers. This paper focuses on one such combination, which uses gravity-compensation together with reinforcement learning (RL). We present a study of the effects of gravity …

Using Taint Analysis And Reinforcement Learning (Tarl) To Repair Autonomous Robot Software, Damian Lyons, Saba Zahra

Faculty Publications

It is important to be able to establish formal performance bounds for autonomous systems. However, formal verification techniques require a model of the environment in which the system operates; a challenge for autonomous systems, especially those expected to operate over longer timescales. This paper describes work in progress to automate the monitor and repair of ROS-based autonomous robot software written for an a-priori partially known and possibly incorrect environment model. A taint analysis method is used to automatically extract the data-flow sequence from input topic to publish topic, and instrument that code. A unique reinforcement learning approximation of MDP utility …

A Comparison Of Contextual Bandit Approaches To Human-In-The-Loop Robot Task Completion With Infrequent Feedback, Matt Mcneill, Damian Lyons

Faculty Publications

Artificially intelligent assistive agents are playing an increased role in our work and homes. In contrast with currently predominant conversational agents, whose intelligence derives from dialogue trees and external modules, a fully autonomous domestic or workplace robot must carry out more complex reasoning. Such a robot must make good decisions as soon as possible, learn from experience, respond to feedback, and rely on feedback only as much as necessary. In this research, we narrow the focus of a hypothetical robot assistant to a room tidying task in a simulated domestic environment. Given an item, the robot chooses where to put …

An Application Of Sliding Mode Control To Model-Based Reinforcement Learning, Aaron Thomas Parisi

Master's Theses

The state-of-art model-free reinforcement learning algorithms can generate admissible controls for complicated systems with no prior knowledge of the system dynamics, so long as sufficient (oftentimes millions) of samples are available from the environ- ment. On the other hand, model-based reinforcement learning approaches seek to leverage known optimal or robust control to reinforcement learning tasks by mod- elling the system dynamics and applying well established control algorithms to the system model. Sliding-mode controllers are robust to system disturbance and modelling errors, and have been widely used for high-order nonlinear system control. This thesis studies the application of sliding mode control …

Robot Navigation In Cluttered Environments With Deep Reinforcement Learning, Ryan Weideman

Master's Theses

The application of robotics in cluttered and dynamic environments provides a wealth of challenges. This thesis proposes a deep reinforcement learning based system that determines collision free navigation robot velocities directly from a sequence of depth images and a desired direction of travel. The system is designed such that a real robot could be placed in an unmapped, cluttered environment and be able to navigate in a desired direction with no prior knowledge. Deep Q-learning, coupled with the innovations of double Q-learning and dueling Q-networks, is applied. Two modifications of this architecture are presented to incorporate direction heading information that …

A Deep Recurrent Q Network Towards Self-Adapting Distributed Microservices Architecture (In Press), Basel Magableh

Articles

One desired aspect of microservices architecture is the ability to self-adapt its own architecture and behaviour in response to changes in the operational environment. To achieve the desired high levels of self-adaptability, this research implements the distributed microservices architectures model, as informed by the MAPE-K model. The proposed architecture employs a multi adaptation agents supported by a centralised controller, that can observe the environment and execute a suitable adaptation action. The adaptation planning is managed by a deep recurrent Q-network (DRQN). It is argued that such integration between DRQN and MDP agents in a MAPE-K model offers distributed microservice architecture …