Robotics Commons^™

Learning Adaptive Control For Safe Collaborative Autonomous Driving, Fabian Alexis Hernandez

Theses and Dissertations

Connected autonomous vehicles improve urban driving through collaborative perception via Vehicle-to-Everything (V2X) communication. Frameworks such as V2Xverse leverage this collaboration for planning and perception, yet their controllers rely on fixed parameters that cannot adapt to varying traffic. Control Barrier Functions (CBFs) enforce safety by constraining actions within a safe set, but a fixed barrier gain imposes a single operating point: conservative settings reduce throughput while permissive settings under-react to hazards.

This research proposes an adaptive CBF framework that learns state-dependent, class-specific barrier gains via constrained Reinforcement Learning (RL). A compact policy outputs separate gains for vehicles, pedestrians, and bicycles, parameterizing …

Digital Replica For Trustworthy Cooperative Autonomous Vehicles, Hady Farahat

Theses and Dissertations

Trust in an automated system can be defined as confidence in a vehicle's reliability, safety, and predictability, which is essential for the acceptance and widespread adoption of fully autonomous vehicles (FAVs); without it, users might disengage from using autonomous vehicles or reject the technology altogether. Most of the previous research has focused on trust from an ego vehicle perspective.

However, next-generation vehicles are becoming more autonomous and connected, relying on vehicle-to-vehicle technology and vehicle-to-infrastructure technology with no human intervention. Hence, trust becomes more complex and fragile as multiple agents interact with each other, and it might become harder to establish …

Qualitative Stereo Vision Using Distributed Extended Waltz Filtering, Ben Mathew

Theses and Dissertations

Stereo vision is a fundamental problem in computer vision, aimed at reconstructing three-dimensional scene structure from two or more two-dimensional images. Traditional stereo algorithms rely on quantitative disparity estimation, often constrained by calibration precision, lighting variations, and surface texture. In contrast, our proposed Qualitative Stereo Vision seeks to understand depth relationships and spatial configurations from multiple planar views through symbolic reasoning and constraint satisfaction, offering a more flexible and cognitively plausible approach to scene interpretation.

This dissertation presents a novel framework called Distributed Extended Waltz Filtering, designed to provide qualitative stereo vision, particularly in the presence of occlusions—a persistent challenge …

Re-Engaging Driver Attention Using Chatgpt: A Multimodal Study On Stress Impact And Driving Performance, Alaa Zaki Elfiqi

Theses and Dissertations

Driving is considered a complex task that requires continuous focus and attention from the driver. Driving performance can also be impacted by changes in the driver’s stress level. However, limited research explores methods to address this challenge. With the current advances in Large Language Models (LLMs) and their ability to engage in human-like interaction, this study investigates the potential of using ChatGPT, designed to speak Egyptian Arabic, in re-engaging driver attention under different driving scenarios for low-stress and high-stress conditions within a virtual reality (VR) environment driving simulator. Drivers were asked to follow a leading car into two scenarios with …

Adaptive Multi-Sensor Fusion For Robust Autonomous Perception In Unstructured Environments, Samantha S. Carley

Theses and Dissertations

Autonomous vehicles commonly employ multiple sensors to perceive their surroundings. Coupling these sensors would ideally improve perception compared to using a single sensor. An autonomous system can be equipped with object localization and classification, often performed using a visual camera to understand a scene intelligently. Object detection and classification can also be applied to LiDAR and infrared (IR) sensors to further enhance scene awareness of the autonomous system. Herein, sensor-level, decision-level, and feature-level fusion are explored to assess their impact on perception and mitigate sensor disagreements. Specifically, the fusing of RGB, LiDAR, and IR sensor data to improve object classification …

Navigating The Future Advancing Autonomous Vehicles Through Robust Target Recognition And Real-Time Avoidance, Mohammed Ahmed Mohammed Hussein

Theses and Dissertations

The problem being tackled by this thesis is a very important one and very relevant to our days and times: it is about making improved target recognition and enhanced real-time response skills in AVs under simulated conditions. Our plan is to put some enhanced sensory capabilities into these vehicles and see if that makes them safer and more reliable. We are using as our base a particular object recognition algorithm (YOLOv7) and a particular simulation environment (CARLA). We utilized the CARLA 0.9.14 simulator on Ubuntu 20.04 as a more stable option than the initially used CARLA 0.9.15 on Ubuntu 22.04, …

Metaheuristic Techniques To Optimize Trajectory Planning Of Uav Swarms: Enhancing Data Acquisition In Wireless Sensor Networks, Nada Ali Mohamed Ahmed Ahmed

Theses and Dissertations

Unmanned aerial vehicles (UAVs) have become increasingly integrated into various applications due to their cost-efficiency, rapid deployment, flexible maneuvers, and enhanced performance. This has led to the development of a new field called UAV-assisted Wireless Sensor Networks (U-WSNs), which focus on data routing, network performance optimization, and planning UAV trajectories between sensor nodes in wireless sensor networks. In this thesis, a new framework has been proposed to manage a swarm of UAVs cooperatively serving large-scale wireless sensor networks. The framework consists of three optimization problems: distributing sensor nodes among UAVs, finding optimal trajectories in the presence of obstacles, and performing …

End-To-End Autonomous Quadcopter Using Reinforcement Learning, Mohamed Marwan Chawa

Theses and Dissertations

This thesis investigates the potential of Reinforcement Learning (RL) for achieving robust and adaptable quadcopter control, focusing on trajectory and attitude stabilization. We compare state-of-the-art RL algorithms, specifically Proximal Policy Optimization (PPO), against traditional Proportional-Integral-Derivative (PID) controllers across three tasks: hovering, slow trajectory following, and fast trajectory following. To enhance realism, we employ a modified PyFlyt simulation environment with a high-fidelity Crazyflie 2.x model, accounting for motor dynamics, noise, wind disturbances, and aerodynamic drag.

The challenge of operating a quadcopter can be divided into two distinct parts: planning a flight path and actually following that path. Our focus is on …

Predicting Vegetation Override Force For Off-Road Autonomy, Marc Nicholas Moore

Theses and Dissertations

Vegetation override is an important aspect of off-road ground vehicle mobility. An autonomous ground vehicle’s (AGV) perception system must distinguish between vegetation that can be easily driven through from vegetation that cannot. Predicting the resistance of vegetation could allow path- planning systems to make this distinction. However, despite its importance, direct measurement of vegetation resistance is rare, as most studies use indirect proprioceptive data, such as inertial measurements, as proxies for override force. Notably, there is a lack of empirical data on the override resistance of small stems (< 2.5 cm) and clusters of vegetation on medium-sized (approx. 1000kg) vehicles. To address this gap, a comprehensive dataset of override measurements was collected for clumps of small vegetation relevant to intermediate-sized AGVs navigating off-road terrain. This dataset includes over 70 recordings using the Robot Operating System (ROS) during controlled driving experiments through small trees, grasses, and bushes. The collected data includes light detection and ranging (LiDAR) scans, imagery, force measurements from integrated load cells, and simultaneous localization and mapping (SLAM) information. A key contribution of this research is the development and calibration of a custom push bar system equipped with load cells to directly measure override forces. These measurements are compared to empirical models previously developed by the U.S. Army Corps of Engineers for larger single-stem vegetation. A preprocessing pipeline was developed to automatically extract and label LiDAR and camera data according to these force measurements. This self-labeled dataset was then used to train machine learning models that predict override resistance of vegetation from LiDAR and camera scans alone. This research characterizes the relationship between override forces and the observable features of vegetation as measured by LiDAR and camera sensors. Deep learning models were developed and trained to predict override forces based on different input modalities and features derived from point clouds and images. The performance of these models was compared across various input features to investigate how deep learning can create a generalizable and accurate force prediction system.

Heterogeneous Collaborative Robotics: Multi-Robot Navigation In Dynamic Environments, Tyler Nicholas Raettig

Theses and Dissertations

Abstract: The challenges of multi-robot navigation in dynamic environments, focusing on uncertainties in obstacle complexities, partial observation, and the transition of policies from simulations to the real world. The proposed approach utilizes a deep reinforcement learning (DRL) framework enabling a Light Detection and Ranging (LiDAR)-equipped robot to communicate with a camera-equipped robot to achieve optimal paths despite their different sensors. The key contributions include the development of a cooperative architecture for information exchange between robots, a DRL-based framework for learning navigation policies, and a training mechanism based on dynamic randomization for enhanced real-world adaptability. Experimental validation using Gazebo simulations demonstrates …

Improving Robustness Of Learning-Based Approaches In Autonomous Systems And Engineering Education, Godwyll Aikins

Theses and Dissertations

This dissertation advances the development of robust learning-based approaches across two complementary domains: engineering education and autonomous systems. Through four studies, this research addresses critical challenges in preparing data-proficient engineers and developing reliable autonomous systems that can operate under uncertainty and incomplete information. The engineering education study examines how mechanical and aerospace engineering undergraduates conceptualize and develop data proficiency skills essential for modern engineering practice. Through interviews with 27 students, the research employs the How People Learn framework to analyze student perspectives on information literacy, data interpretation, and computational thinking. The findings inform pedagogical strategies for developing data proficiency in …

End-To-End Learning For A Low-Cost Robotics Arm, Abhishek Chothani

Theses and Dissertations

Robotic manipulation is a cornerstone of automation, with the ultimate goal of developing versatile systems capable of executing a wide range of real-world tasks autonomously. Traditional robotics approaches, while reliable and widely adopted in industrial settings, often struggle with adaptability, perception, and dynamic task execution. This thesis explores the evolution from classical robotics techniques to modern learning-based approaches, leveraging advancements in artificial intelligence to overcome these limitations.

Initially, the thesis presents a pick-and-place pipeline built using the Drake robotics framework and the KUKA iiwa robotic arm. This system employs a pseudoinverse controller for inverse kinematics to perform structured tasks like …

Real-Time Robot Pose Estimation For Industry 4.0: Enhancing Motion Validation And Monitoring With A Vision-Based Framework, Jad Samaha

Theses and Dissertations

Amidst the era of Industry 4.0, robots became integral to automated production lines by performing complex tasks with high precision and adapting to changing production needs in real-time. Therefore, ensuring their precise and reliable operation is paramount for maintaining high-quality standards and operational efficiency. This invoked the need for an automated motion validation tool that guarantees accurate program task execution, by detecting deviations or anomalies that may indicate mechanical faults or software errors. Given the exponential advancements in AI, particularly in Computer Vision, a vision-based solution is ideal for ensuring the correct positioning of a robot in real-time. Operating independently …

Uav-Based Tracking And Following Of Railroad Lines, Keith Michael Lewandowski

Theses and Dissertations

Given the pivotal role of the railroad industry in modern transportation and the potential risks associated with track malfunctions, the inspection and maintenance of railroad tracks emerges as a critical concern. While existing solutions excel in performing accurate measurements and detection, they often rely on large, expensive, and time-consuming platforms for inspections. This project, however, seeks to solve the same problem with the use of an unmanned aerial vehicle (UAV), significantly reducing time and cost while maintaining detection capabilities. In particular, this solution is ideal for large-scale, high-level inspections following major events such as floods [6], hurricanes [7] or earthquakes …

Development Of The Structure And Control System Of A Stewart Platform Robot For Human Balance Recovery Interventions, Rhobenn R. Alvarez Zambrano

Theses and Dissertations

In this thesis, the process to design a Stewart platform parallel robot for balance recovery with given assembly constraints and mobility requirements is described. A Model Based Design (MBD) approach in MATLAB was used as a tool to model and optimize the design of the platform through quick and repeatable workspace and movements simulations. An algorithm based on Inverse Kinematics was used to find the most adequate Stewart platform configuration which yields a workspace that fulfills the design goals best. Solidworks was used as a 3D CAD Modeling tool to elaborate machining blueprints while ensuring that each piece fits accurately …

Navigating The Rules: Integrating Td3 And Sensor Fusion For Traffic-Aware Autonomous Vehicle Path Planning, Mahmoud Ayman Mohamed Elsayed

Theses and Dissertations

This work presents a novel algorithm for local path planning for autonomous vehicles (AVs) which prioritizes both safety and adherence to traffic regulations, addressing critical functions for AV navigation, such as navigating complex environments, avoiding obstacles, and ensuring passenger and road users safety. The algorithm integrates the Twin Delayed Deep Deterministic Policy Gradient (TD3) with sensor fusion based on Nvidia Convolutional Neural Network (NCNN). The study utilizes the CARLA simulator, and real-world datasets, including KITTI and WAYMO, to train and evaluate the proposed algorithm. The proposed algorithm leverages the complementary strengths of Imitation Learning (IL) and Deep Reinforcement Learning (DRL) …

Mobile Robot Path Planning Optimization Problem Using Multi- Objective Genetic Algorithm, Suresh Ks

Theses and Dissertations

Mobile Robot Path Planning problem (MRPPP) is the most prominent research area employed in different real-time environments. The research domain of robotics offers abundant opportunities for researchers in various engineering fields with different dimensions. The automation of physical movements of the robots with intelligence to make dynamic decisions for interacting with the environment opens up a lot of challenges to the research community.

A variety of approaches are employed to solve the Mobile Robot Path Planning Problem (MRPP), which is to derive a feasible collision-free path to reach the destination from the given starting point by avoiding obstacles. The solution …

Particle Swarm Optimization For Training Quadrotor Pid Controller, Eric Xavier Rodriguez

Theses and Dissertations

The objective of this research is to establish a fundamental approach to tuning PID (Proportional-Integral-Derivative) parameters for a simulated quadrotor drone. Implementing a PID controller for autonomous flight provides a straightforward and efficient method for monitoring and correcting robotic movement based on the robot's current state. However, applying a PID approach to a quadrotor's flight controller poses challenges, such as assigning multiple parameters to control an inherently under-actuated system. This includes the need to find optimal parameter values that reduce the likelihood of large overshoots and lengthy adjustment times. Ineffectively tuning PID parameters can have detrimental effects on autonomously …

Developing Resilient Defense Strategies Against Pheromone-Based Attacks In Foraging Robot Swarms, Ryan A. Luna

Theses and Dissertations

This thesis delves into the security of stochastic pheromone-based foraging algorithms within swarm robotic systems, a subset of foraging algorithms distinguished by their reliance on probabilistic decision-making mechanisms inspired by the natural world. Such algorithms face vulnerabilities in stigmergic communication that threaten to disrupt swarm operations. This research investigates these vulnerabilities, presenting two distinct contributions.

The first contribution examines the implementation of quarantine strategies as a defensive measure to isolate and mitigate the impact of fake resource attacks. By simulating these attacks, this study quantitatively assesses their detrimental effects on swarm efficiency and explores the efficacy …

Efficient Connectivity Management And Path Planning For Iot And Uav Networks, Amirahmad Chapnevis

Theses and Dissertations

This dissertation explores how to better manage resources in mobile networks, especially for enhancing the performance of Unmanned Aerial Vehicles (UAV)-supported IoT networks. We explored ways to set up a flexible communication architecture that can handle large IoT deployments by making good use of mobile core network resources like bearers and data paths. We developed strategies that meet the needs of IoT networks and enhance network performance. We also developed and tested a system that combines traffic from several mobile devices that use the same user identity and network resources within the core mobile network. We used everyday smartphones, SIM …

Shape Inverse Prediction Of Magnetic Field-Actuated Soft Robots By Neural Network Machine Learning, Lineth J. Perez Monsalve

Theses and Dissertations

Soft robotics has drawn tremendous interest in recent years because the compliance and motion of soft robotics enable biocompatibility and versatility for many applications, such as human-machine interaction, wearable and assistive devices, and health monitoring. This study introduces a novel predictive modeling approach using neural networks for shape control of magnetic soft robots. The robots are made of silicone materials embedded with hard magnetic particles, which respond to the external magnetic field provided by a ring-type of permanent magnet. These robots, free from physical connections to external devices, i.e., non-tethered actuation, hold significant potential for applications in healthcare, such as …

Exploring End-User Environments For The Control And Programming Of Collaborative Robots, Luiz Felipe Fronchetti Dias

Theses and Dissertations

To collaborate with the ongoing development of robotics, this thesis highlights three research contributions to collaborative robot programming. The first study evaluates block-based programming as an alternative for two-armed robots. A commercial solution is put in contrast with a block-based programming language. Both programming solutions are evaluated by 52 participants in an experiment involving a pick-and-place task. This study brings insights into human-robot collaboration, including robot positioning and interaction challenges. The second study discusses using mixed-reality devices as a potential workaround to the manual positioning of industrial and collaborative robots. Five different control interfaces implemented in mixed reality were used …

External Runtime Execution Monitoring Of A Cyber Physical System Via Trace Interfaces, Peter Vaughan Truslow

Theses and Dissertations

In the past two decades, Unmanned Aerial Systems have progressed from expensive military hardware or one-off custom builds, to include off-the-shelf drones that can be purchased for a rather affordable price and flown by nearly anyone. As the technology and performance have improved, the door is opened to applications that require operation in environments where the consequences for failure are high, such as operating in the navigable airspace or in urban environments, or with human passengers. This requires a great deal of trust in the reliability and integrity of the control systems of the aircraft. A method of monitoring the …

Safety-Aware Autonomous Robot Navigation, Mapping And Control By Optimization Techniques, Tingjun Lei

Theses and Dissertations

The realm of autonomous robotics has seen impressive advancements in recent years, with robots taking on essential roles in various sectors, including disaster response, environmental monitoring, agriculture, and healthcare. As these highly intelligent machines continue to integrate into our daily lives, the pressing imperative is to elevate and refine their performance, enabling them to adeptly manage complex tasks with remarkable efficiency, adaptability, and keen decision-making abilities, all while prioritizing safety-aware navigation, mapping, and control systems. Ensuring the safety-awareness of these robotic systems is of paramount importance in their development and deployment. In this research, bio-inspired neural networks, nature-inspired intelligence, deep …

Accelerating Machine Learning Inference For Satellite Component Feature Extraction Using Fpgas., Andrew Ekblad

Theses and Dissertations

Running computer vision algorithms requires complex devices with lots of computing power, these types of devices are not well suited for space deployment. The harsh radiation environment and limited power budgets have hindered the ability of running advanced computer vision algorithms in space. This problem makes running an on-orbit servicing detection algorithm very difficult. This work proposes using a low powered FPGA to accelerate the computer vision algorithms that enable satellite component feature extraction. This work uses AMD/Xilinx’s Zynq SoC and DPU IP to run model inference. Experiments in this work centered around improving model post processing by creating implementations …

Skeleton-Based Hand Gesture Recognition Using Data-Level Fusion, Oluwaleke Yusuf

Theses and Dissertations

Hand Gesture Recognition (HGR) is a form of perceptual computing that allows artificial systems to capture and interpret human gestures. HGR has applications in human-machine interaction, virtual reality, augmented reality, and human behavior analysis. The human hand can assume a near-infinite number of poses and orientations to form myriad gestures, thus increasing the difficulty of the HGR task.

The hand skeleton of connected joints effectively describes the hand’s geometric shape and thus contains richer semantic gesture information while eliminating noise from individual differences in physical hand characteristics. The efficacy and computational efficiency of skeleton-based HGR frameworks can be significantly enhanced …

Initiating Change In Care: Socially Assistive Robots, Sooraj Sushama

Theses and Dissertations

Socially assistive robots (SAR) are autonomous machines equipped with sensors and software that allow them to interact socially with humans. SAR robots are commonly used in healthcare settings to provide patients with non-clinical support, such as conversation and emotional companionship. SARs can also deliver reminders, monitor vital signs, and provide educational information about health conditions or medications. Researchers have studied SAR applications in detail. Additionally, there has been prior research on SAR where users' sociodemographic factors and technology acceptance were studied. But even though the backbone of SAR is an advanced technology, no known research has been done on users' …

Portable Robotic Navigation Aid For The Visually Impaired, Lingqiu Jin

Theses and Dissertations

This dissertation aims to address the limitations of existing visual-inertial (VI) SLAM methods - lack of needed robustness and accuracy - for assistive navigation in a large indoor space. Several improvements are made to existing SLAM technology, and the improved methods are used to enable two robotic assistive devices, a robot cane, and a robotic object manipulation aid, for the visually impaired for assistive wayfinding and object detection/grasping. First, depth measurements are incorporated into the optimization process for device pose estimation to improve the success rate of VI SLAM's initialization and reduce scale drift. The improved method, called depth-enhanced visual-inertial …

Real-Time Motion Controller For Human-Robot Teams Utilizing Artificial Potential Fields, Gabriella Graziani

Theses and Dissertations

This thesis presents a real-time robotic motion control system for human-robot teams. The framework utilizes artificial potential fields (APFs) to guide robotic agents towards a “goal” agent while navigating around “obstacle” agents; these goals and obstacles are also dynamic agents with their own set of tasks. This system is also developed for a live-programming environment, where a controlling agent updates the tasks of all agents within the system at any time during the system’s runtime. This motion controller was created and tested for a human-robot choreographic team. After the controller was fully integrated on a Trossen Robotic LoCobot Wx200 robotic …

Improving The Flexibility And Robustness Of Machine Tending Mobile Robots, Richard Ethan Hollingsworth

Theses and Dissertations

While traditional manufacturing production cells consist of a fixed base robot repetitively performing tasks, the Industry 5.0 flexible manufacturing cell (FMC) aims to bring Autonomous Industrial Mobile Manipulators (AIMMs) to the factory floor. Composed of a wheeled base and a robot arm, these collaborative robots (cobots) operate alongside people while autonomously performing tasks at different workstations. AIMMs have been tested in real production systems, but the development of the control algorithms necessary for automating a robot that is a combination of two cobots remains an open challenge before the large scale adoption of this technology occurs in industry. Currently popular …