Engineering Commons^™

Reinforcement-Learning-Based Adaptive Tracking Control For A Space Continuum Robot Based On Reinforcement Learning, Da Jiang, Zhiqin Cai, Zhongzhen Liu, Haijun Peng, Zhigang Wu

Journal of System Simulation

Abstract: Aiming at the tracking control for three-arm space continuum robot in space active debris removal manipulation, an adaptive sliding mode control algorithm based on deep reinforcement learning is proposed. Through BP network, a data-driven dynamic model is developed as the predictive model to guide the reinforcement learning to adjust the sliding mode controller's parameters online, and finally realize a real-time tracking control. Simulation results show that the proposed data-driven predictive model can accurately predict the robot's dynamic characteristics with the relative error within ±1% to random trajectories. Compared with the fixed-parameter sliding mode controller, the proposed adaptive controller …

Reinforcement Learning-Based Cooperative Optimal Output Regulation Via Distributed Adaptive Internal Model, Weinan Gao, Mohammed Mynuddin, Donald C. Wunsch, Zhong Ping Jiang

Electrical and Computer Engineering Faculty Research & Creative Works

In this article, a data-driven distributed control method is proposed to solve the cooperative optimal output regulation problem of leader-follower multiagent systems. Different from traditional studies on cooperative output regulation, a distributed adaptive internal model is originally developed, which includes a distributed internal model and a distributed observer to estimate the leader's dynamics. Without relying on the dynamics of multiagent systems, we have proposed two reinforcement learning algorithms, policy iteration and value iteration, to learn the optimal controller through online input and state data, and estimated values of the leader's state. By combining these methods, we have established a basis …

Learning To Play An Imperfect Information Card Game Using Reinforcement Learning, Buğra Kaan Demi̇rdöver, Ömer Baykal, Ferdanur Alpaslan

Turkish Journal of Electrical Engineering and Computer Sciences

Artificial intelligence and machine learning are widely popular in many areas. One of the most popular ones is gaming. Games are perfect testbeds for machine learning and artificial intelligence with various scenarios and types. This study aims to develop a self-learning intelligent agent to play the Hearts game. Hearts is one of the most popular trick-taking card games around the world. It is an imperfect information card game. In addition to having a huge state space, Hearts offers many extra challenges due to its nature. In order to ease the development process, the agent developed in the scope of this …

Low-Reynolds-Number Locomotion Via Reinforcement Learning, Yuexin Liu

Dissertations

This dissertation summarizes computational results from applying reinforcement learning and deep neural network to the designs of artificial microswimmers in the inertialess regime, where the viscous dissipation in the surrounding fluid environment dominates and the swimmer’s inertia is completely negligible. In particular, works in this dissertation consist of four interrelated studies of the design of microswimmers for different tasks: (1) a one-dimensional microswimmer in free-space that moves towards the target via translation, (2) a one-dimensional microswimmer in a periodic domain that rotates to reach the target, (3) a two-dimensional microswimmer that switches gaits to navigate to the designated targets in …

Effective Resource Scheduling For Collaborative Computing In Edge-Assisted Internet Of Things Systems, Qianqian Wang

Electronic Thesis and Dissertation Repository

Along with rapidly evolving communications technologies and data analytics, Internet of Things (IoT) systems interconnect billions of smart devices to gather, exchange, analyze data, and perform tasks autonomously, which poses a huge pressure on IoT devices' computing capabilities. Taking advantage of collaborative computing enabled by cloud computing and edge computing technologies, IoT devices can offload computation tasks to idle computing devices and remote servers, thus alleviating their pressure. However, scheduling resources effectively to realize collaborative computing remains a severe challenge due to diverse application objectives, limited distributed resources, and unpredictable environments. To overcome the above challenges, this thesis aims to …

Advancing The Radiation Oncology Clinic With Motion Management And Automatic Treatment Planning, Damon Anton Sprouts

Bioengineering Dissertations

The leading cause of premature death (death under the age of 70) is cancer. The top five cancers for both male and female are: lung, colorectum, pancreas, breast cancer, and prostate. In 2020 there was an estimated 19.3 million new cases with an estimated 9.9 million deaths. The cancer burden is expected to grow to 28.4 million by the year 2040. Surgery, chemotherapy, and radiotherapy are the three pillars in the modern clinic for cancer treatment. In radiotherapy, ionizing radiation particles can travel through the patient body, deposit energy along the way and damage the DNA Structure. There needs to …

Application Of Improved Q Learning Algorithm In Job Shop Scheduling Problem, Yejian Zhao, Yanhong Wang, Jun Zhang, Hongxia Yu, Zhongda Tian

Journal of System Simulation

Abstract: Aiming at the job shop scheduling in a dynamic environment, a dynamic scheduling algorithm based on an improved Q learning algorithm and dispatching rules is proposed. The state space of the dynamic scheduling algorithm is described with the concept of "the urgency of remaining tasks" and a reward function with the purpose of "the higher the slack, the higher the penalty" is disigned. In view of the problem that the greedy strategy will select the sub-optimal actions in the later stage of learning, the traditional Q learning algorithm is improved by introducing an action selection strategy based on the …

Outdoor Operations Of Multiple Quadrotors In Windy Environment, Deepan Lobo

Dissertations

Coordinated multiple small unmanned aerial vehicles (sUAVs) offer several advantages over a single sUAV platform. These advantages include improved task efficiency, reduced task completion time, improved fault tolerance, and higher task flexibility. However, their deployment in an outdoor environment is challenging due to the presence of wind gusts. The coordinated motion of a multi-sUAV system in the presence of wind disturbances is a challenging problem when considering collision avoidance (safety), scalability, and communication connectivity. Performing wind-agnostic motion planning for sUAVs may produce a sizeable cross-track error if the wind on the planned route leads to actuator saturation. In a multi-sUAV …

Efficient Deep Learning And Its Applications, Zi Wang

Doctoral Dissertations

Deep neural networks (DNNs) have achieved huge successes in various tasks such as object classification and detection, image synthesis, game-playing, and biological developmental system simulation. State-or-the-art performance on these tasks is usually achieved by designing deeper and wider DNNs with the cost of huge storage size and high computational complexity. However, the over-parameterization problem of DNNs constrains their deployment in resource-limited devices, such as drones and mobile phones.

With these concerns, many network compression approaches are developed, such as quantization, neural architecture search, network pruning, and knowledge distillation. These approaches reduce the sizes and computational costs of DNNs while maintaining …

Pulse-Coupled Oscillator Networks: Achieving Phase Continuity And Learning Optimal Control In Physical Systems, Timothy Anglea

All Dissertations

In this dissertation, we consider the application of pulse-coupled oscillator theory to real-world, physical networks. When the phase of an oscillator is associated with a physical measure, such as clock timing or robotic heading, discontinuous adjustments of the oscillator's phase is undesirable and potentially disadvantageous. Rather, continuous adjustment of the oscillator phase value is needed over a certain amount of time. To ensure that both synchronization and desynchronization can still be achieved under the constraint of continuous phase value changes, we pursue a novel approach to analyze the generalization of a pulse-coupled oscillator network with a time-varying coupling strength. We …

A Deep Reinforcement Learning Approach With Prioritized Experience Replay And Importance Factor For Makespan Minimization In Manufacturing, Jose Napoleon Martinez

LSU Doctoral Dissertations

In this research, we investigated the application of deep reinforcement learning (DRL) to a common manufacturing scheduling optimization problem, max makespan minimization. In this application, tasks are scheduled to undergo processing in identical processing units (for instance, identical machines, machining centers, or cells). The optimization goal is to assign the jobs to be scheduled to units to minimize the maximum processing time (i.e., makespan) on any unit.

Machine learning methods have the potential to "learn" structures in the distribution of job times that could lead to improved optimization performance and time over traditional optimization methods, as well as to adapt …

Distributed Control And Learning Of Connected And Autonomous Vehicles Approaching And Departing Signalized Intersections, Joshua Onyeka Ogbebor

LSU Master's Theses

This thesis outlines methods for achieving energy-optimal control policies for autonomous vehicles approaching and departing a signalized traffic intersection. Connected and autonomous vehicle technology has gained wide interest from both research institutions and government agencies because it offers immense promise in advancing efficient energy usage and abating hazards that beset the current transportation system. Energy minimization is itself crucial in reducing the greenhouse emissions from fossil-fuel-powered vehicles and extending the battery life of electric vehicles which are presently the major alternative to fossil-fuel-powered vehicles. Two major forms of fuel minimization are studied. First, the eco-driving problem is solved for a …

Research On The Construction Method Of Simulation Evaluation Index Of Operation Effectiveness Operation Concept Traction, Ziwei Zhang, Liang Li, Zhiming Dong, Yifei Wang, Li Duan

Journal of System Simulation

Abstract: Agents are difficult to be directly modeled and simulated due to the complexity of their own interaction and learning behaviors. Aiming at the common problems in the discrete simulation of the agent, the event transfer mechanism of the discrete event system specification (DEVS) atomic model is applied to express the interaction and learning of an agent. Through the interaction mode of the agent, the transfer control of multi-state external events, the port connection mode, as well as the introduction of reinforcement learning event transfer representation, a discrete simulation construction method of the agent based on the DEVS atomic model …

Multiagent Routing Problem With Dynamic Target Arrivals Solved Via Approximate Dynamic Programming, Andrew E. Mogan

Theses and Dissertations

This research formulates and solves the multiagent routing problem with dynamic target arrivals (MRP-DTA), a stochastic system wherein a team of autonomous unmanned aerial vehicles (AUAVs) executes a strike coordination and reconnaissance (SCAR) mission against a notional adversary. Dynamic target arrivals that occur during the mission present the team of AUAVs with a sequential decision-making process which we model via a Markov Decision Process (MDP). To combat the curse of dimensionality, we construct and implement a hybrid approximate dynamic programming (ADP) algorithmic framework that employs a parametric cost function approximation (CFA) which augments a direct lookahead (DLA) model via a …

Team Air Combat Using Model-Based Reinforcement Learning, David A. Mottice

Theses and Dissertations

We formulate the first generalized air combat maneuvering problem (ACMP), called the MvN ACMP, wherein M friendly AUCAVs engage against N enemy AUCAVs, developing a Markov decision process (MDP) model to control the team of M Blue AUCAVs. The MDP model leverages a 5-degree-of-freedom aircraft state transition model and formulates a directed energy weapon capability. Instead, a model-based reinforcement learning approach is adopted wherein an approximate policy iteration algorithmic strategy is implemented to attain high-quality approximate policies relative to a high performing benchmark policy. The ADP algorithm utilizes a multi-layer neural network for the value function approximation regression mechanism. One-versus-one …

Reinforcement Learning For Process Control: Applications To Energy Systems, Elijah Ballard Hedrick

Graduate Theses, Dissertations, and Problem Reports

Reinforcement learning (RL) is a machine learning method that has recently seen significant research activity owing to its successes in the areas of robotics and gameplaying (Silver et al., 2017). However, significant challenges exist in the extension of these control methods to process control problems, where state and input signals are nearly always continuous and more stringent performance guarantees are required. The goal of this work is to explore ways that modern RL algorithms can be adapted to handle process control problems; avenues for this work include using RL with existing controllers such as model predictive control (MPC) and adapting …

Knowledge-Infused Reinforcement Learning, Kaushik Roy, Manas Gaur, Qi Zhang, Amit Sheth

Publications

Virtual health agents (VHAs) have received considerable attention, but the early focus has been on collecting data, helping patients follow generic health guidelines, and providing reminders for clinical appointments. While presenting the collected data and frequency of visits to the clinician is useful, further context and personalization are needed for a VHA to interpret and understand what the data means in clinical terms. This has made their use in managing health limited. Such understanding enables patient empowerment and self-appraisal – i.e., aiding the patient in interpreting the data to understand the changes in the patient’s health conditions, and self-management – …

Developing Reactive Distributed Aerial Robotics Platforms For Real-Time Contaminant Mapping, Joshua Ashley

Theses and Dissertations--Electrical and Computer Engineering

The focus of this research is to design a sensor data aggregation system and centralized sensor-driven trajectory planning algorithm for fixed-wing aircraft to optimally assist atmospheric simulators in mapping the local environment in real-time. The proposed application of this work is to be used in the event of a hazardous contaminant leak into the atmosphere as a fleet of sensing unmanned aerial vehicles (UAVs) could provide valuable information for evacuation measures. The data aggregation system was designed using a state-of-the-art networking protocol and radio with DigiMesh and a process/data management system in the ROS2 DDS. This system was tested to …