Robotics Commons^™

Development And Testing Of A Computer Vision Pose Estimation System For Planar Mobile Robots, Andrew Jones

Master's Theses

This thesis details the development and testing of a computer vision-based real-time pose estimation system for differential drive robots. A single camera with a fisheye lens is used to locate ArUco markers placed at fixed locations and attached to robots. Using the OpenCV library, the Perspective-n-Point (PnP) problem is solved to facilitate the transformation of 2-D robot positions in an image to world-frame coordinates. An analytical solution is presented to estimate robot poses based on a single PnP solution, rather than solving the PnP problem for each pose estimate. Pose information is relayed to individual robots using a multi-microcontroller architecture …

Evaluation Of General Voronoi Diagram Decomposition For Harmonic Fields In Navigation Of Dynamic Environments, Franco Abullarade

Master's Theses

Harmonic potential fields provide provably minimum-free navigation, but any change to the workspace geometry invalidates the field and forces a costly global recomputation, typically restricting them to static environments. This thesis extends the harmonic map framework of Vlantis et al., which maps the free workspace onto a unit disk and uses an atlas of per-region transformations, to dynamic indoor settings. First, we replace their manually annotated room partition with an automatic decomposition based on the Generalized Voronoi Diagram, allowing the atlas to be built from an arbitrary occupancy grid in an automated way. Second, we introduce a localized repair procedure …

Deep Multimodal Sensor Fusion For Autonomous Driving: A Real-World Deployment On Little Blue, Raphael T. Pamie-George

Electrical & Computer Engineering Theses & Dissertations

Accurate perception and control systems are essential for autonomous vehicles to safely interpret and navigate complex environments. Individual sensing modalities such as cameras, LiDAR, and inertial measurement units (IMUs) provide valuable environmental and vehicle state information; however, each sensor exhibits inherent limitations when used independently. Camera performance can degrade under varying lighting conditions, LiDAR measurements may be sparse or affected by environmental noise, and IMU measurements are susceptible to drift over time. Little Blue, a research autonomous vehicle platform from ODU (Old Dominion University), previously relied primarily on camera-based data for neural network training, limiting the system’s ability to exploit …

Evaluating Machine Learning Models On Classification Of Novel Cyber Attacks In The Healthcare Domain, Promise Ehimen

Dissertations, Theses, and Projects

The increasing adoption of the Internet of Medical Things (IoMT) has improved healthcare delivery through connected medical devices while simultaneously expanding the cybersecurity risks facing healthcare organizations. Although machine learning based intrusion detection systems have demonstrated high detection accuracy, their ability to respond reliably to previously unseen cyberattacks remains uncertain. This study investigated how a Neural Network model and a Logistic Regression model classified novel cyberattacks within the IoMT environment. The Neural Network and Logistic Regression models were both trained and tested using a subset of the CICIoMT2024 benchmark dataset. The Neural Network achieved 99.82% test accuracy and a 0.94 …

Design And Modeling Of A Robotic Manipulator For Agricultural Explorations, Trent Thomas Powell

Graduate Theses and Dissertations

This thesis focuses on the modeling, design, and early testing of a spherical revolute revolute prismatic (RRP) robotic manipulator for probe based applications. The system is motivated by agricultural grain probing, where a probe is positioned over a truck bed and inserted into the grain to collect samples as trucks enter storage facilities, ports, hubs, or transfer points. A spherical RRP configuration is useful for this type of task because it provides a large accessible workspace while maintaining a relatively simple mechanical structure. A model based design approach is used to develop the system, beginning with forward and inverse kinematics …

Sam-1: An Assistive Robot For Remote Patient Monitoring, Aana Farhana Ahsan

USF Tampa Graduate Theses and Dissertations

The global aging population has increased the demand for caregiving, while the availability of caregivers has not grown at the same pace. These trends have contributed to disparities in consistent and accessible care for older adults and increased caregiver burden. Geriatric care robots have emerged as a promising solution to support older adults and reduce caregiver burden. Existing systems have been developed for a wide range of applications, including rehabilitation, social support, assistance with activities of daily living, therapy, and caregiver telepresence. This thesis presents the development of SAM-1, an assistive robot designed to support remote monitoring of older adults …

Vision-Guided Motion Planning For Autonomous Catch With A Robotic Arm, Kayla Go-Oco, Ashik K. Islam, Raegan Gritzmacher

Electrical Engineering

Modern robotic systems are often evaluated using static and highly controlled tasks, such as pick-and-place demonstrations, which do not fully represent the uncertainty and adaptability required in real-world environments. A major challenge in robotics research is enabling robotic systems to perceive, track, and respond to dynamic objects in real time while maintaining accurate and reliable motion control. This project addresses these challenges by developing an autonomous robotic platform in which an OpenMANIPULATOR-Y robotic arm detects, tracks, and attempts to catch a moving tennis ball using a vision-guided control system. The system integrates YOLO-based object detection with an Intel RealSense D455 …

Autonomous Vision-Based Litter Collection Rover, Dante Michael Benedetti, Benjamin Scott Tavares, Nathan Heil

Electrical Engineering

This report documents the design, implementation, and testing of an autonomous litter-collection rover developed as a Senior Project Design Lab (EE 460/463/464) at California Polytechnic State University. The rover integrates autonomy, computer vision, embedded real-time control, mecanum-wheel omnidirectional mobility, and a two-degree-of-freedom robotic arm to detect, approach, and collect small ground-level litter such as bottles, wrappers, and paper fragments.

The system uses a two-layer compute architecture: an NVIDIA Jetson Orin Nano running ROS 2 for perception, SLAM, and path planning, paired with an STM32L4A6ZG microcontroller for real-time motor control and odometry. The robot is built on a multi-level aluminum frame …

Integration Of Robotic Arm And Conveyor With Programmable Logic Controller, Drake A. Small

Master's Theses

To meet new curriculum demands brought on by Cal Poly's upcoming switch to semesters, a new, cross-disciplinary lab module was developed for EE 435 (Industrial Power Control and Automation). The module emphasizes career-applicable skills, preparing students for the field of controls engineering within the manufacturing industry. These skills include robotic control, computer networking and configuration, and embedded systems. The work focused on integrating a collaborative robot arm to pick-and-place boxes on a conveyor. A myCobot 320 Pi and an Ultimation Powered Roller MDR Conveyor were integrated with the existing PLC system using Modbus RTU and EtherNet/IP communication protocols, respectively. A …

Design And Parametric Study Of A Mems-Based Reservoir Computer For Reinforcement Learning, Andrew P. Carr

Master's Theses

Single-node reservoir computing (RC) is a hardware-efficient approach to machine learning, leveraging the dynamics of physical systems. In this work, two reinforcement learning algorithms, Q-learning and Proximal Policy Optimization (PPO), are applied to a simulated micro-electro-mechanical system (MEMS)-based reservoir computer to solve both discrete and continuous control tasks. MEMS-based reservoirs are low-power, compact, and their natural frequencies (kHz to MHz) pair well with real-time control loops. To explore the relationship between reservoir dynamics and learning performance, a parametric study is conducted on two reservoir hyperparameters, reservoir size and neuron separation, using CartPole-v1 and MountainCar-v0. The RC successfully learns multiple tasks …

Adaptive Multimodal Smart Home Control On A Raspberry Pi 5 Using Hand Gestures, Voice Cues, And User Feedback, Vaibhav Bora

Theses

A real time multimodal smart home control system deployed on a Raspberry Pi 5 is presented. The system combines hand gestures, short voice cues, and proximity aware interaction to execute household commands such as light brightness control, fan speed adjustment, and stop or kill switch actions. Lightweight gesture and keyword spotting voice classifiers were trained offline and exported to TensorFlow Lite for efficient on device inference. For more natural spoken phrases, the system additionally integrates a locally deployed pretrained offline ASR component rather than a speech recognizer trained from scratch. Using a USB camera and microphone, the system operates fully …

Simulated Intelligence, Surveillance, And Play, Atienah Cape

Theses - ALL

This MFA thesis explores modern panopticism and the feeling of being watched by creating and installing Four Real Friends, an interactive robotic artwork inspired by the 1998 Furby. The project looks at how surveillance technologies are not just outside forces, but part of daily life, often feeling familiar, playful, and even social. The installation uses hardware design, custom circuits, computer vision, and real-time data display to collect and process how visitors interact in the gallery. Its nostalgic and friendly look encourages people to take part, while still showing how data is collected and displayed as it happens. This makes surveillance …

Symmetries Of Legged Locomotion: Theory And Applications In Animals And Robots, Jiayu Ding

Dissertations - ALL

Legged locomotion in animals and robots is naturally modeled as a hybrid dynamical system: smooth body evolution is repeatedly interrupted by discrete contact events such as touchdown and liftoff. Although symmetry is a powerful organizing principle in smooth dynamical systems, a comparably systematic symmetry framework for hybrid legged locomotion remains underdeveloped. This dissertation develops such a framework for a scoped class of periodic hybrid locomotion models and uses it to relate theoretical gait analysis, animal locomotion studies, and robotic gait generation. The dissertation first extends symmetry notions from autonomous dynamics to hybrid legged systems and distinguishes symmetries of the hybrid …

Designing A Collaborative Uav-Ugv System For Autonomous Challenge Completion, Ember Reysen

Honors Theses

The coordination between Unmanned Aerial Vehicles (UAVs) and Unmanned Ground Vehicles (UGVs) holds significant potential for completing complex tasks such as disaster response and construction surveillance by leveraging the strengths of aerial flight and stable on-ground transportation. Unfortunately, coordination between UAVs and UGVs is extremely complex and task-dependent, leading to limited understanding of autonomous collaboration. This thesis explores the process of designing an UAV-UGV collaborative system capable of aerially scouting a target destination, relaying its coordinates to a ground vehicle, and performing a precision landing for unison travel. I utilize computer vision techniques via OpenCV for target marker identification, ROS …

Autonomous Deficiency Detection And Vision-Language Summarization For Underground Infrastructure On Embedded Edge Systems, Johny Lopez

LSU New Orleans Theses and Dissertations

Aging underground infrastructure poses significant risks to public health and environmental safety, yet structural condition assessment remains bottlenecked by labor-intensive manual CCTV inspections. This thesis proposes a comprehensive algorithmic framework enabling fully autonomous, real-time deficiency detection, geometric assessment, and natural language reporting on resource- constrained edge computing platforms. Three core components address this challenge. First, RAPID-SCAN, a novel semantic segmentation architecture utilizing a Dynamic Feature Pyramid Network and Channel-Spatial Attention, achieves real-time, pixel-precise defect localization with dramatically reduced parameters. Second, an Edge-Optimized Vision-Language Model pipeline employing LoRA and 4-bit QLoRA quantization compresses Phi-3.5 for local deployment, en- abling autonomous technical …

Stretch Web Teleoperation Keyboard Final Report, Aaron Chou, Francisco Irazaba

Computer Engineering

This project added keyboard controls to the Hello Robot Stretch Web Teleop interface. The goal was to make the robot easier to control from a web browser without needing a gamepad or separate terminal-based keyboard program. The new interface allows users to control different parts of the robot with customizable key bindings.

Learning Adaptive Control For Safe Collaborative Autonomous Driving, Fabian Alexis Hernandez

Theses and Dissertations

Connected autonomous vehicles improve urban driving through collaborative perception via Vehicle-to-Everything (V2X) communication. Frameworks such as V2Xverse leverage this collaboration for planning and perception, yet their controllers rely on fixed parameters that cannot adapt to varying traffic. Control Barrier Functions (CBFs) enforce safety by constraining actions within a safe set, but a fixed barrier gain imposes a single operating point: conservative settings reduce throughput while permissive settings under-react to hazards.

This research proposes an adaptive CBF framework that learns state-dependent, class-specific barrier gains via constrained Reinforcement Learning (RL). A compact policy outputs separate gains for vehicles, pedestrians, and bicycles, parameterizing …

Safe Control Design For Quadruped Locomotion In Unstructured Environments Using Linear Transfer Operators, Sriram Sundar Krishnamoorthy Shankara Narayanan

All Dissertations

Deploying quadruped robots in unstructured, obstacle-rich environments requires control and planning methods that remain safe and reliable despite complex terrain geometry, limited sensing, and inevitable modeling errors. This thesis develops operator-theoretic tools for safe control design of robotic systems using linear transfer operators, with a focus on quadruped locomotion in unstructured environments. The central goal is to develop a unified operator-theoretic framework for safe control design based on the Perron–Frobenius (P–F) and Koopman operators. In particular, the thesis leverages \emph{density functions} to develop safe navigation frameworks in the dual space of densities. In the operator-theoretic perspective, the P–F operator governs …

Temporal Logic Planning In Semantic Maps Of Unknown Environments Using Tl-Rrt, Dongrui Yang

McKelvey School of Engineering Graduate Student Theses & Dissertations

Autonomous mobile robots are increasingly expected to perform complex missions in unstructured environments. Traditional path planning approaches handle simple point-to-point navigation, but struggle with complex tasks that involve temporal and logical orderings of objectives. Linear Temporal Logic (LTL) provides a method for complex missions (e.g., sequential visits to multiple targets or surveillance tasks) in a strict way. This thesis presents an integrated planning framework that enables a robot to satisfy LTL-based task specifications in an unknown environment by combining a Temporal Logic RRT* (TL-RRT*) planner with semantic mapping. The robot builds a semantic map of its environment online using simultaneous …

A Low-Cost Motion Classification System For A Stuffed Animal Using An Imu And Machine Learning, Rachel N. Guynes

Honors Theses

One of the many fields that has seen the integration of robots is therapy. Zoomorphic robots (ZR) are designed to look and behave like animals to assist in Animal Assisted Therapy (AAT) practices. Studies show that ZRs can provide benefits similar to working with an actual animal; however, their high cost limits their accessibility. This thesis documents the process of building a real-time, low-cost motion classification system that can be attached to a stuffed animal to make it more interactive. Using a Random Forest (RF) classifier, the system identifies movements with approximately 81.67% accuracy.

Multiple-Valued Quantum Automata For Robotics, Yuchen Huang

Dissertations and Theses

This dissertation introduces a new type of quantum automata, their encoding and circuit realization. I concentrate on possible applications in robotics. Several methods and application of quantum automata and quantum circuit-based controllers for elementary robotic systems, with a focus on humanoid robot motion, emotion, and behavior generation are illustrated in detail. The research introduces several novel methodologies that bridge quantum computing principles with robotic control, aiming to overcome the limitations of classical deterministic and probabilistic approaches.

The dissertation first presents a quantum-circuit-based framework for generating non-repetitive and expressive (e)motions in a humanoid robot actor, using superposition and entanglement to produce …

Optimizing Auv Perception For Competitive Underwater Robotics: Analyzing Performance And Simplicity Of Machine Vision Hardware, Joseph King

Honors Theses

Underwater robotics is a growing field with many practical applications, such as pipeline and offshore structure monitoring, deep sea mining, mine reconnaissance/removal, and marine environmental monitoring. USM’s Robotics Club will participate in the international RoboSub competition, where student-led teams build autonomous underwater vehicles (AUVs) that perform a variety of tasks within a pool requiring camera-based object detection. This project analyzes affordable hardware and software options for providing machine vision to USM’s future RoboSub AUV. The performance of an ESP32-CAM microcontroller, Raspberry Pi 5, and Jetson Orin Nano single-board computers running FOMO and YOLO object detection models was compared. These models …

Behavioral-Centric Team Evaluation Via Consistent Rewards, Clement Kudakwashe Nyanhongo

Dartmouth College Ph.D Dissertations

Across human domains ranging from sports to business and organizational settings, complex tasks are often solved by teams rather than individuals, leveraging benefits such as interaction, mutual support, complementary skills, cohesion, and task allocation. Evaluating team effectiveness, however, is inherently challenging due to the subjectivity of many existing techniques and the limitations of outcome-driven metrics that primarily focus on performance scores while overlooking the team processes that generated the scores. To address these challenges, this dissertation proposes a behavioral-centric, end-to-end framework for team evaluation grounded in reward functions that model sequential team behavior. Reward functions offer compact and interpretable representations …

Tuning And Performance Of Pid Controlled Low Complexity Systems, Timothy Evans

Honors Theses

Proportional integral derivative (PID) controllers are used for precise position and orientation control in systems such as autonomous underwater vehicles (AUVs). This project supports the University of Southern Mississippi’s (USM) Robotics Club’s RoboSub AUV effort by developing, troubleshooting, and manually tuning PID controllers to characterize tracking performance and settling time across systems of increasing complexity. Initially, the project hypothesized that tracking performance would be reduced and settling times would increase as system complexity advanced from one degree-of-freedom (DOF) to two DOF. However, prior research was found that suggests that for small disturbances around an equilibrium state, separate PID-controlled DOFs can …

A Pretraining-Based Framework For On-Device Training Of Imu-Based Locomotion Mode Detection For Wearable Active Exoskeletons, Muhammad Tahir Khan

LSU Master's Theses

Active exoskeletons are being developed to support human movement in physically demanding industries such as construction. For these systems to work effectively, they must be able to correctly identify the user’s current activity. This process is known as locomotion mode detection and plays an important role in selecting the appropriate control parameters for exoskeletons. Many existing approaches use inertial measurement units (IMUs) to recognize these activities and have shown strong performance. However, most of these methods depend on large amounts of labeled data collected under specific conditions. As a result, they often do not perform well when applied to new …

Identification Of Thruster Faults In Underwater Vehicles By Using Custom Encodings In Spiking Neural Networks, Donovan Gegg

LSU Master's Theses

Autonomous Underwater Vehicles (AUVs) are untethered robotic platforms used for tasks such as seafloor mapping, infrastructure inspection, and environmental monitoring. Recent technological advances have produced smaller, more affordable platforms, broadening access to research teams and small companies alike. This miniaturization comes at the cost of them handling drawbacks associated with a more compact machine such as reduced battery capacity as well as limited processing and sensing capabilities. These constraints make small-sized marine vehicle’s reliability critical as they can cause malfunctions, making the loss of a vehicle more likely. Actuator faults are particularly consequential as unintended and unstable control in an …

A Capability Maturity Model For Artificial Intelligence Integration In Supply Chain Management, Lordt Becklines

Dissertations

Artificial Intelligence (AI) is transforming Supply Chain Management (SCM), yet many organizations struggle to assess their readiness for AI adoption and to understand how AI capabilities develop across maturity stages. This dissertation addresses this gap by developing a Capability Maturity Model (CMM) for AI integration in SCM, grounded in Organizational Information Processing Theory (OIPT), the Resource-Based View, and related capability frameworks. The model provides a structured approach for evaluating an organization's information-processing requirements, resource configurations, and alignment needed for effective AI-enabled supply chain operations.

Using a design science research approach, the AI-SCM CMM and its associated assessment instrument were derived …

Digital Replica For Trustworthy Cooperative Autonomous Vehicles, Hady Farahat

Theses and Dissertations

Trust in an automated system can be defined as confidence in a vehicle's reliability, safety, and predictability, which is essential for the acceptance and widespread adoption of fully autonomous vehicles (FAVs); without it, users might disengage from using autonomous vehicles or reject the technology altogether. Most of the previous research has focused on trust from an ego vehicle perspective.

However, next-generation vehicles are becoming more autonomous and connected, relying on vehicle-to-vehicle technology and vehicle-to-infrastructure technology with no human intervention. Hence, trust becomes more complex and fragile as multiple agents interact with each other, and it might become harder to establish …

Robotic Air Hockey Table, William Forcey, Andrew Piunno, Kaden Carpenter, Xander Zavatchen

Williams Honors College, Honors Research Projects

Air hockey, a popular arcade game, is traditionally designed for two players. This limits the game’s accessibility for individuals who wish to practice or enjoy it as a single player. To solve this problem, a robotic system was implemented to play air hockey against a human player. The speed and acceleration of the puck and mallet were measured from a game played between humans to inform the required movement capabilities of the robot. The robotic opponent implemented observes the location of the puck on the table using a camera and predicts where it will be in the future. A Cartesian …

End-To-End Development And Experimental Validation Of A 1/10-Scale Autonomous Vehicle, Rikkin Pankaj Panchal

Electrical Engineering Theses

Autonomous vehicle development demands vast resources, making scaled down platforms a critical alternative for solving core algorithmic challenges. The primary contribution of this thesis is the end to end development and validation of a complete real time autonomous driving pipeline deployed on a one tenth scale vehicle. To streamline platform development, an AI assisted annotation framework automates dataset generation, significantly reducing manual labor while improving training data quality. The system perception stack features a reinforcement learning guided online multi camera calibration framework that enables adaptive surround view stitching without the need for offline recalibration. This is paired with robust lane …