Engineering Commons^™

A New Cache Replacement Policy In Named Data Network Based On Fib Table Information, Mehran Hosseinzadeh, Neda Moghim, Samira Taheri, Nasrin Gholami

VMASC Publications

Named Data Network (NDN) is proposed for the Internet as an information-centric architecture. Content storing in the router’s cache plays a significant role in NDN. When a router’s cache becomes full, a cache replacement policy determines which content should be discarded for the new content storage. This paper proposes a new cache replacement policy called Discard of Fast Retrievable Content (DFRC). In DFRC, the retrieval time of the content is evaluated using the FIB table information, and the content with less retrieval time receives more discard priority. An impact weight is also used to involve both the grade of retrieval …

Research And Development Of Simulation Training Platform For Multi-Agent Collaborative Decision-Making, Cheng Cheng, Zhijie Chen, Ziming Guo, Ni Li

Journal of System Simulation

Abstract: Reinforcement learning simulation platform can be an interactive and training environment for reinforcement learning. In order to make the simulation platform compatible with the multi-agent reinforcement learning algorithms and meet the needs of simulation in military field, the similar processes in multi-agent reinforcement learning algorithms are refined and a unified interface is designed to embed and verify different types of deep reinforcement learning algorithms on the simulation platform and to optimize the back-end service of the simulation platform to accelerate the training process of the algorithm model. The experimental results show that, by unifing the interface, the simulation platform …

Location Aware Task Offloading Framework For Edge Computing Empowered By Reconfigurable Intelligent Surfaces, Md Sahabul Hossain

Electrical and Computer Engineering ETDs

In this thesis, an energy efficient task offloading mechanism in a multi-access edge computing (MEC) environment is introduced, based on the principles of Contract Theory. The technology of Reconfigurable Intelligent Surfaces (RIS) is adopted and serves as the enabler for the energy efficient task offloading, from the perspective of location-awareness and improved communication environment. Initially a novel positioning, navigation, and timing solution is designed, based on the RIS technology and an artificial intelligent method that selects a set of RISs to perform the multilateration technique and determine the Internet of Things (IoT) nodes’ positions in an efficient and accurate manner …

Adaptive Personalized Drug Delivery Method For Warfarin And Anemia Management: Modeling And Control., Affan Affan

Electronic Theses and Dissertations

Personalized precision medicine aims to develop the appropriate treatments for suitable patients at the right time to obtain optimal results. Personalized medicine is challenging due to inter- and intra-patient variability, narrow therapeutic window, the effect of other medications, comorbidity (more than one disease at a time), nonlinear patient dynamics, and time-varying patient dose response characteristics which include bleeding (internal and external). This research aims to develop a framework for an adaptive personalized modeling and control method with minimum clinical patient specific dose response data for optimal drug dosing. The proposed methodology is applied to anemia and warfarin management. It is …

Deep Reinforcement Learning For The Design Of Structural Topologies, Nathan Brown

All Dissertations

Advances in machine learning algorithms and increased computational efficiencies have given engineers new capabilities and tools for engineering design. The presented work investigates using deep reinforcement learning (DRL), a subset of deep machine learning that teaches an agent to complete a task through accumulating experiences in an interactive environment, to design 2D structural topologies. Three unique structural topology design problems are investigated to validate DRL as a practical design automation tool to produce high-performing designs in structural topology domains.

The first design problem attempts to find a gradient-free alternative to solving the compliance minimization topology optimization problem. In the proposed …

Neural Airport Ground Handling, Yaoxin Wu, Jianan Zhou, Yunwen Xia, Xianli Zhang, Zhiguang Cao, Jie Zhang

Research Collection School Of Computing and Information Systems

Airport ground handling (AGH) offers necessary operations to flights during their turnarounds and is of great importance to the efficiency of airport management and the economics of aviation. Such a problem involves the interplay among the operations that leads to NP-hard problems with complex constraints. Hence, existing methods for AGH are usually designed with massive domain knowledge but still fail to yield high-quality solutions efficiently. In this paper, we aim to enhance the solution quality and computation efficiency for solving AGH. Particularly, we first model AGH as a multiple-fleet vehicle routing problem (VRP) with miscellaneous constraints including precedence, time windows, …

Intercell Dynamic Scheduling Method Based On Deep Reinforcement Learning, Jing Ni, Mengke Ma

Journal of System Simulation

Abstract: In order to solve the intercell scheduling problem of dynamic arrival of machining tasks and realize adaptive scheduling in the complex and changeable environment of the intelligent factory, a scheduling method based on a deep Q network is proposed. A complex network with cells as nodes and workpiece intercell machining path as directed edges is constructed, and the degree value is introduced to define the state space with intercell scheduling characteristics. A compound scheduling rule composed of a workpiece layer, unit layer, and machine layer is designed, and hierarchical optimization makes the scheduling scheme more global. Since double deep …

Uav-Enabled Task Offloading Strategy For Vehicular Edge Computing Networks, Feng Hu, Haiyang Gu, Jun Lin

Journal of System Simulation

Abstract: As intelligent vehicles are equipped with more and more sensors, the explosive growth of sensor data is generated, which brings severe challenges to vehicular communication and computing. In addition, the modern road presents a three-dimensional structure, and the system architecture of traditional vehicular networks cannot guarantee full coverage and seamless computing. A task offloading strategy for UAV-assisted and 6G-enabled (Sixth Generation) vehicular edge computing networks is proposed. Furthermore, a flexible and intelligent vehicular edge computing mode is composed by vehicles and UAVs, which provide three-dimensional edge computing services for delay-sensitive and computation-intensive vehicular tasks, and ensure timely processing and …

Imitative Generation Of Optimal Guidance Law Based On Reinforcement Learning, Zhengxuan Jia, Tingyu Lin, Yingying Xiao, Guoqiang Shi, Hao Wang, Bi Zeng, Yiming Ou, Pengpeng Zhao

Journal of System Simulation

Abstract: Under the background of high-speed maneuvering target interception, an optimal guidance law generation method for head-on interception independent of target acceleration estimation is proposed based on deep reinforcement learning. In addition, its effectiveness is verified through simulation experiments. As the simulation results suggest, the proposed method successfully achieves head-on interception of high-speed maneuvering targets in 3D space and largely reduces the requirement for target estimation with strong uncertainty, and it is more applicable than the optimal control method.

Aircraft Assignment Method For Optimal Utilization Of Maintenance Intervals, Runxia Guo, Yifu Wang

Journal of System Simulation

Abstract: The aircraft assignment problem is studied from a maintenance assurance perspective. In order to ensure its continuous airworthiness, civil aircraft are required to perform maintenance tasks, i. e., scheduled inspections, at specified intervals. The scheduled inspection interval is usually controlled by the number of flight cycles (FC), flight hours (FH), or flight days (FD), whichever comes first. In order to make balanced use of the inspection interval, an aircraft assignment model for a given fleet size is developed to optimize the maintenance interval utilization, and it is solved by a reinforcement learning algorithm to minimize the variance of the …

Dynamic Influence Diagram-Based Deep Reinforcement Learning Framework And Application For Decision Support For Operators In Control Rooms, Joseph Mietkiewicz, Ammar N. Abbas, Chidera Winifred Amazu, Anders L. Madsen, Gabriele Baldissone

Articles

In today’s complex industrial environment, operators are often faced with challenging situations that require quick and accurate decision-making. The human-machine interface (HMI) can display too much information, leading to information overload and potentially compromising the operator’s ability to respond effectively. To address this challenge, decision support models are needed to assist operators in identifying and responding to potential safety incidents. In this paper, we present an experiment to evaluate the effectiveness of a recommendation system in addressing the challenge of information overload. The case study focuses on a formaldehyde production simulator and examines the performance of an improved Human-Machine Interface …

Imitation Improvement Learning For Large-Scale Capacitated Vehicle Routing Problems, The Viet Bui, Tien Mai

Research Collection School Of Computing and Information Systems

Recent works using deep reinforcement learning (RL) to solve routing problems such as the capacitated vehicle routing problem (CVRP) have focused on improvement learning-based methods, which involve improving a given solution until it becomes near-optimal. Although adequate solutions can be achieved for small problem instances, their efficiency degrades for large-scale ones. In this work, we propose a newimprovement learning-based framework based on imitation learning where classical heuristics serve as experts to encourage the policy model to mimic and produce similar or better solutions. Moreover, to improve scalability, we propose Clockwise Clustering, a novel augmented framework for decomposing large-scale CVRP into …

Model-Driven Analysis Of Ecg Using Reinforcement Learning, Christian O'Reilly, Sai Durga Rithvik Oruganti, Deepa Tilwani, Jessica Bradshaw

Publications

Modeling is essential to better understand the generative mechanisms responsible for experimental observations gathered from complex systems. In this work, we are using such an approach to analyze the electrocardiogram (ECG). We present a systematic framework to decompose ECG signals into sums of overlapping lognormal components. We use reinforcement learning to train a deep neural network to estimate the modeling parameters from an ECG recorded in babies from 1 to 24 months of age. We demonstrate this model-driven approach by showing how the extracted parameters vary with age. From the 751,510 PQRST complexes modeled, 82.7% provided a signal-to-noise ratio that …

Sim-To-Real Reinforcement Learning Framework For Autonomous Aerial Leaf Sampling, Ashraful Islam

Department of Computer Science and Engineering: Dissertations, Theses, and Student Research

Using unmanned aerial systems (UAS) for leaf sampling is contributing to a better understanding of the influence of climate change on plant species, and the dynamics of forest ecology by studying hard-to-reach tree canopies. Currently, multiple skilled operators are required for UAS maneuvering and using the leaf sampling tool. This often limits sampling to only the canopy top or periphery. Sim-to-real reinforcement learning (RL) can be leveraged to tackle challenges in the autonomous operation of aerial leaf sampling in the changing environment of a tree canopy. However, trans- ferring an RL controller that is learned in simulation to real UAS …

Accelerating The Derivation Of Optimal Powertrain Control Strategies Using Reinforcement Learning And Virtual Prototypes, Daniel Egan

All Dissertations

The push for improvements in fuel economy while reducing tailpipe emissions has resulted in significant increases in automotive powertrain complexity, subsequently increasing the resources, both time and money, needed to develop them. Powertrain performance is heavily influenced by the quality of their controller/calibration with modern powertrains reaching levels of complexity where using traditional design of experiment-based methodologies to develop them can take years. Recently, reinforcement learning (RL), a machine learning technique, has emerged as a method to rapidly create optimal controllers for systems of unlimited complexity directly which creates an opportunity to use RL to reduce the overall time and …

Using Actor-Critic Reinforcement Learning For Control Of A Quadrotor Dynamics, Edgar Adrian Torres

Theses and Dissertations

This paper presents a quadrotor controller using reinforcement learning to generate near-optimal control signals. Two actor-critic algorithms are trained to control quadrotor dynamics. The dynamics are further simplified using small angle approximation. The actor-critic algorithm’s control policy is derived from Bellman’s equation providing a sufficient condition to optimality. Additionally, a smoother converter is implemented into the trajectory providing more reliable results. This paper provides derivations to the quadrotor’s dynamics and explains the control using the actor-critic algorithm. The results and simulations are compared to solutions from a commercial, optimal control solver, called DIDO.

Near-Optimal Control Of A Quadcopter Using Reinforcement Learning, Alberto Velazquez-Estrada

Theses and Dissertations

This paper presents a novel control method for quadcopters that achieves near-optimal tracking control for input-affine nonlinear quadcopter dynamics. The method uses a reinforcement learning algorithm called Single Network Adaptive Critics (SNAC), which approximates a solution to the discrete-time Hamilton-Jacobi-Bellman (DT-HJB) equation using a single neural network trained offline. The control method involves two SNAC controllers, with the outer loop controlling the linear position and velocities (position control) and the inner loop controlling the angular position and velocities (attitude control). The resulting quadcopter controller provides optimal feedback control and tracks a trajectory for an infinite-horizon, and it is compared with …

Research On Unmanned Swarm Combat System Adaptive Evolution Model Simulation, Zhiqiang Li, Yuanlong Li, Laixiang Yin, Xiangping Ma

Journal of System Simulation

Abstract: Aiming at the fact that the intelligent unmanned swarm combat system is mainly composed of large-scale combat individuals with limited behavioral capabilities and has limited ability to adapt to the changes of battlefield environment and combat opponents, a learning evolution method combining genetic algorithm and reinforcement learning is proposed to construct an individual-based unmanned bee colony combat system evolution model. To improve the adaptive evolution efficiency of bee colony combat system, an improved genetic algorithm is proposed to improve the learning and evolution speed of bee colony individuals by using individual-specific mutation optimization strategy. Simulation experiment on …

Multi-Agent Cooperative Combat Simulation In Naval Battlefield With Reinforcement Learning, Ding Shi, Xuefeng Yan, Lina Gong, Jingxuan Zhang, Donghai Guan, Mingqiang Wei

Journal of System Simulation

Abstract: Due to the rapidly-changed situations of future naval battlefields, it is urgent to realize the high-quality combat simulation in naval battlefields based on artificial intelligence to comprehensively optimize and improve the combat effectiveness of our army and defeat the enemy. The collaboration of combat units is the key point and how to realize the balanced decision-making among multiple agents is the first task. Based on decoupling priority experience replay mechanism and attention mechanism, a multi-agent reinforcement learning-based cooperative combat simulation (MARL-CCSA) network is proposed. Based on the expert experience, a multi-scale reward function is designed, on which a naval …

Deep Reinforcement Learning For Articulatory Synthesis In A Vowel-To-Vowel Imitation Task, Denis Shitov, Elena Pirogova, Tadeusz A. Wysocki, Margaret Lech

Department of Electrical and Computer Engineering: Faculty Publications

Articulatory synthesis is one of the approaches used for modeling human speech production. In this study, we propose a model-based algorithm for learning the policy to control the vocal tract of the articulatory synthesizer in a vowel-to-vowel imitation task. Our method does not require external training data, since the policy is learned through interactions with the vocal tract model. To improve the sample efficiency of the learning, we trained the model of speech production dynamics simultaneously with the policy. The policy was trained in a supervised way using predictions of the model of speech production dynamics. To stabilize the training, …

Dqn-Based Joint Scheduling Method Of Heterogeneous Tt&C Resources, Naiyang Xue, Dan Ding, Yutong Jia, Zhiqiang Wang, Yuan Liu

Journal of System Simulation

Abstract: Joint scheduling of heterogeneous TT&C resources as research object, a deep Q network (DQN) algorithm based on reinforcement learning is proposed. The characteristics of the joint scheduling problem of heterogeneous TT&C resources being fully analyzied and mathematical language being used to describe the constraints affecting the solution, a resource joint scheduling model is established. From the perspective of applying reinforcement learning, two neural networks with the same structure and the action selection strategies based onεgreedy algorithm are respectively designed after Markov decision process description, and DQN solution framework is established. The simulation results show that DQN-based heterogeneous …

Optimizing Constraint Selection In A Design Verification Environment For Efficient Coverage Closure, Vanessa Cooper

CCE Theses and Dissertations

No abstract provided.

Power System Dynamic Control And Performance Improvement Based On Reinforcement Learning, Wei Gao

Electronic Theses and Dissertations

This dissertation investigates the feasibility and effectiveness of using Reinforcement Learning (RL) techniques for power system dynamic control, particularly voltage and frequency control. The conventional control strategies used in power systems are complex and time-consuming due to the complicated high-order nonlinearities of the system. RL, which is a type of neural network-based technique, has shown promise in solving these complex problems by fitting any nonlinear system with the proper network structure.

The proposed RL algorithm, called Guided Surrogate Gradient-based Evolution Strategy (GSES) determines the weights of the policy (which generates the action for our control reference signal) without back-propagation process …

Extended Kalman Filter Based Resilient Formation Tracking Control Of Multiple Unmanned Vehicles Via Game-Theoretical Reinforcement Learning, Lei Xue, Bei Ma, Jian Liu, Chaoxu Mu, Donald C. Wunsch

Electrical and Computer Engineering Faculty Research & Creative Works

In This Paper, We Discuss the Resilient Formation Tracking Control Problem of Multiple Unmanned Vehicles (MUV). a Dynamic Leader-Follower Distributed Control Structure is Utilized to Optimize the Performance of the Formation Tracking. for the Follower of the MUV, the Leader is a Cooperative Unmanned Vehicle, and the Target of Formation Tracking is a Non-Cooperative Unmanned Vehicle with a Nonlinear Trajectory. Therefore, an Extended Kalman Filter (EKF) Observer is Designed to Estimate the State of the Target. Then the Leader of the MUV is Adjusted Dynamically According to the State of the Target. in Order to Describe the Interactions between the …

Continual Optimal Adaptive Tracking Of Uncertain Nonlinear Continuous-Time Systems Using Multilayer Neural Networks, Irfan Ganie, S. (Sarangapani) Jagannathan

Electrical and Computer Engineering Faculty Research & Creative Works

This study provides a lifelong integral reinforcement learning (LIRL)-based optimal tracking scheme for uncertain nonlinear continuous-time (CT) systems using multilayer neural network (MNN). In this LIRL framework, the optimal control policies are generated by using both the critic neural network (NN) weights and single-layer NN identifier. The critic MNN weight tuning is accomplished using an improved singular value decomposition (SVD) of its activation function gradient. The NN identifier, on the other hand, provides the control coefficient matrix for computing the control policies. An online weight velocity attenuation (WVA)-based consolidation scheme is proposed wherein the significance of weights is derived by …

Dynamic Influence Diagram-Based Deep Reinforcement Learning Framework And Application For Decision Support For Operators In Control Rooms, Joseph Mietkiewicz, Ammar N. Abbas, Chidera Winifred Amazu, Anders L. Madsen, Gabriele Baldissone

Conference papers

In today’s complex industrial environment, operators are often faced with challenging situations that require quick and accurate decision-making. The human-machine interface (HMI) can display too much information, leading to information overload and potentially compromising the operator’s ability to respond effectively. To address this challenge, decision support models are needed to assist operators in identifying and responding to potential safety incidents. In this paper, we present an experiment to evaluate the effectiveness of a recommendation system in addressing the challenge of information overload. The case study focuses on a formaldehyde production simulator and examines the performance of an improved Human-Machine Interface …

Reinforcement-Learning-Based Adaptive Tracking Control For A Space Continuum Robot Based On Reinforcement Learning, Da Jiang, Zhiqin Cai, Zhongzhen Liu, Haijun Peng, Zhigang Wu

Journal of System Simulation

Abstract: Aiming at the tracking control for three-arm space continuum robot in space active debris removal manipulation, an adaptive sliding mode control algorithm based on deep reinforcement learning is proposed. Through BP network, a data-driven dynamic model is developed as the predictive model to guide the reinforcement learning to adjust the sliding mode controller's parameters online, and finally realize a real-time tracking control. Simulation results show that the proposed data-driven predictive model can accurately predict the robot's dynamic characteristics with the relative error within ±1% to random trajectories. Compared with the fixed-parameter sliding mode controller, the proposed adaptive controller …

Reinforcement Learning-Based Cooperative Optimal Output Regulation Via Distributed Adaptive Internal Model, Weinan Gao, Mohammed Mynuddin, Donald C. Wunsch, Zhong Ping Jiang

Electrical and Computer Engineering Faculty Research & Creative Works

In this article, a data-driven distributed control method is proposed to solve the cooperative optimal output regulation problem of leader-follower multiagent systems. Different from traditional studies on cooperative output regulation, a distributed adaptive internal model is originally developed, which includes a distributed internal model and a distributed observer to estimate the leader's dynamics. Without relying on the dynamics of multiagent systems, we have proposed two reinforcement learning algorithms, policy iteration and value iteration, to learn the optimal controller through online input and state data, and estimated values of the leader's state. By combining these methods, we have established a basis …

Learning To Play An Imperfect Information Card Game Using Reinforcement Learning, Buğra Kaan Demi̇rdöver, Ömer Baykal, Ferdanur Alpaslan

Turkish Journal of Electrical Engineering and Computer Sciences

Artificial intelligence and machine learning are widely popular in many areas. One of the most popular ones is gaming. Games are perfect testbeds for machine learning and artificial intelligence with various scenarios and types. This study aims to develop a self-learning intelligent agent to play the Hearts game. Hearts is one of the most popular trick-taking card games around the world. It is an imperfect information card game. In addition to having a huge state space, Hearts offers many extra challenges due to its nature. In order to ease the development process, the agent developed in the scope of this …

Low-Reynolds-Number Locomotion Via Reinforcement Learning, Yuexin Liu

Dissertations

This dissertation summarizes computational results from applying reinforcement learning and deep neural network to the designs of artificial microswimmers in the inertialess regime, where the viscous dissipation in the surrounding fluid environment dominates and the swimmer’s inertia is completely negligible. In particular, works in this dissertation consist of four interrelated studies of the design of microswimmers for different tasks: (1) a one-dimensional microswimmer in free-space that moves towards the target via translation, (2) a one-dimensional microswimmer in a periodic domain that rotates to reach the target, (3) a two-dimensional microswimmer that switches gaits to navigate to the designated targets in …