Engineering Commons^™

Machine Learning Based Applications For Data Visualization, Modeling, Control, And Optimization For Chemical And Biological Systems, Yan Ma

LSU Doctoral Dissertations

This dissertation report covers Yan Ma’s Ph.D. research with applicational studies of machine learning in manufacturing and biological systems. The research work mainly focuses on reaction modeling, optimization, and control using a deep learning-based approaches, and the work mainly concentrates on deep reinforcement learning (DRL). Yan Ma’s research also involves with data mining with bioinformatics. Large-scale data obtained in RNA-seq is analyzed using non-linear dimensionality reduction with Principal Component Analysis (PCA), t-Distributed Stochastic Neighbor Embedding (t-SNE), and Uniform Manifold Approximation and Projection (UMAP), followed by clustering analysis using k-Means and Hierarchical Density-Based Spatial Clustering with Noise (HDBSCAN). This report focuses …

Ip-Traffic Classification Model Based On Machine Learning Ways, Madjit Malikovich Karimov, Sherzod Rajaboevich Gulomov

Chemical Technology, Control and Management

This article analyzes machine learning ways for classifying traffic filtering: supervised learning and unsupervised learning; semi-supervised learning; reinforcement learning- classification and their use for segmentation and recognition of objects. A method of teaching with supervised learning for filtering traffic is applied, and a classifier training model with supervised learning is proposed.

Intelligent Data-Driven Energy Flow Controllers For Renewable Energy And Electrified Transportation Systems, Juan Rafael Nunez Forestieri

LSU Doctoral Dissertations

In recent years, large scale deployments of electrical energy generation using renewable sources (RES) such as wind, solar and ocean wave power, along with more sustainable means of transformation have emerged in response to different initiatives oriented toward reducing greenhouse gas emissions. Strategies facilitating the integration of renewable generation into the grid and electric propulsion in transportation systems are proposed in this work.

Chapter 2 investigates the grid-connected operation of a wave energy converter (WEC) along with a hybrid supercapacitor/undersea energy storage system (HESS). A combined sizing and energy management strategy (EMS) based on reinforcement learning (RL) is proposed. Comparisons …

Towards Real-Time Reinforcement Learning Control Of A Wave Energy Converter, Enrico Anderlini, Salman Husain, Gordon Parker, Mohammad Abusara, Giles Thomas

Michigan Tech Publications

The levellised cost of energy of wave energy converters (WECs) is not competitive with fossil fuel-powered stations yet. To improve the feasibility of wave energy, it is necessary to develop effective control strategies that maximise energy absorption in mild sea states, whilst limiting motions in high waves. Due to their model-based nature, state-of-the-art control schemes struggle to deal with model uncertainties, adapt to changes in the system dynamics with time, and provide real-time centralised control for large arrays of WECs. Here, an alternative solution is introduced to address these challenges, applying deep reinforcement learning (DRL) to the control of WECs …

Deep Reinforcement Learning Approach To Solve Dynamic Vehicle Routing Problem With Stochastic Customers, Waldy Joe, Hoong Chuin Lau

Research Collection School Of Computing and Information Systems

In real-world urban logistics operations, changes to the routes and tasks occur in response to dynamic events. To ensure customers’ demands are met, planners need to make these changes quickly (sometimes instantaneously). This paper proposes the formulation of a dynamic vehicle routing problem with time windows and both known and stochastic customers as a route-based Markov Decision Process. We propose a solution approach that combines Deep Reinforcement Learning (specifically neural networks-based TemporalDifference learning with experience replay) to approximate the value function and a routing heuristic based on Simulated Annealing, called DRLSA. Our approach enables optimized re-routing decision to be generated …

Gpu Resource Optimization And Scheduling For Shared Execution Environments, Ryan Seamus Luley

Dissertations - ALL

General purpose graphics processing units have become a computing workhorse for a variety of data- and compute-intensive applications, from large supercomputing systems for massive data analytics to small, mobile embedded devices for autonomous vehicles. Making effective and efficient use of these processors traditionally relies on extensive programmer expertise to design and develop kernel methods which simultaneously trade off task decomposition and resource exploitation. Often, new architecture designs force code refinements in order to continue to achieve optimal performance. At the same time, not all applications require full utilization of the system to achieve that optimal performance. In this case, the …

Joint Optimization Control Of Energy Storage System Management And Demand Response, Xueying Gao, Tang Hao, Gangzhong Miao, Zhaowu Ping

Journal of System Simulation

Abstract: The joint optimization problem of energy management and demand response were studied in order to reduce the long-run cost of electricity users equipped with energy storage unit and smart applications, and to increase their benefits meanwhile. The goals were achieved by controlling both the energy storage unit (charging, discharging, or idle) and the load service (access or delay). Based on the random nature of solar photovoltaic, load demand electricity and electricity price, the joint optimization problem was modeled as infinite-horizon Markov decision process model, and Q-learning algorithm was proposed to find the optimal solution. Simulation results show that the …

Noisy Importance Sampling Actor-Critic: An Off-Policy Actor-Critic With Experience Replay, Miriam A M Capretz, Norman Tasfi

Electrical and Computer Engineering Publications

This paper presents Noisy Importance Sampling Actor-Critic (NISAC), a set of empirically validated modifications to the advantage actor-critic algorithm (A2C), allowing off-policy reinforcement learning and increased performance. NISAC uses additive action space noise, aggressive truncation of importance sample weights, and large batch sizes. We see that additive noise drastically changes how off-sample experience is weighted for policy updates. The modified algorithm achieves an increase in convergence speed and sample efficiency compared to both the on-policy actor-critic A2C and the importance weighted off-policy actor-critic algorithm. In comparison to state-of-the-art (SOTA) methods, such as actor-critic with experience replay (ACER), NISAC nears the …

Analysis And Optimization Of The Action Chain Mechanism In Agent2d Underlying In Robocup2d Soccer League, Chen Bing, Feifan Xu, Hanyan Xu, Zekai Cheng, Liu Cheng

Journal of System Simulation

Abstract: In the RoboCup2D soccer league, Agent2D is one of the most widely used underlying team in China. Data transmission noise and the incomplete action chain mechanism make the underlying teams using Agent2D be lack of flexibility. This paper introduces an action correcting parameter and optimizes the operation of the action chain by reinforcement learning mechanism. The performance of the Agent2D underlying team is improved in the game and the adaptability of the team is enhanced. Simulation experiment results show that this method has a certain effect.

Monte Carlo Tree Search Applied To A Modified Pursuit/Evasion Scotland Yard Game With Rendezvous Spaceflight Operation Applications, Joshua A. Daughtery

Theses and Dissertations

This thesis takes the Scotland Yard board game and modifies its rules to mimic important aspects of space in order to facilitate the creation of artificial intelligence for space asset pursuit/evasion scenarios. Space has become a physical warfighting domain. To combat threats, an understanding of the tactics, techniques, and procedures must be captured and studied. Games and simulations are effective tools to capture data lacking historical context. Artificial intelligence and machine learning models can use simulations to develop proper defensive and offensive tactics, techniques, and procedures capable of protecting systems against potential threats. Monte Carlo Tree Search is a bandit-based …

Using Taint Analysis And Reinforcement Learning (Tarl) To Repair Autonomous Robot Software, Damian Lyons, Saba Zahra

Faculty Publications

It is important to be able to establish formal performance bounds for autonomous systems. However, formal verification techniques require a model of the environment in which the system operates; a challenge for autonomous systems, especially those expected to operate over longer timescales. This paper describes work in progress to automate the monitor and repair of ROS-based autonomous robot software written for an a-priori partially known and possibly incorrect environment model. A taint analysis method is used to automatically extract the data-flow sequence from input topic to publish topic, and instrument that code. A unique reinforcement learning approximation of MDP utility …

Hierarchical Multiagent Reinforcement Learning For Maritime Traffic Management, Arambam James Singh, Akshat Kumar, Hoong Chuin Lau

Research Collection School Of Computing and Information Systems

Increasing global maritime traffic coupled with rapid digitization and automation in shipping mandate developing next generation maritime traffic management systems to mitigate congestion, increase safety of navigation, and avoid collisions in busy and geographically constrained ports (such as Singapore's). To achieve these objectives, we model the maritime traffic as a large multiagent system with individual vessels as agents, and VTS (Vessel Traffic Service) authority as a regulatory agent. We develop a hierarchical reinforcement learning approach where vessels first select a high level action based on the underlying traffic flow, and then select the low level action that determines their future …

A Reinforcement Learning Approach To Sequential Acceptance Sampling As A Critical Success Factor For Lean Six Sigma, Hani A. Khalil

Theses and Dissertations

In the 21st century, globalization coupled with technological advancement and free trade has created competition among various businesses enterprises. This competition has led many businesses to adopt various management techniques such as acceptance sampling aimed at transforming their processes in order to remain competitive in the global market and adapt to new market demands. The successful implementation of acceptance sampling is highly dependent on what the academic literature refers to as acceptance sampling optimization. A literature review on the optimization of acceptance sampling has not shown any work that studied whether acceptance sampling and machine learning (ML) plans can be …

The Heuristic Dynamic Programming Approach In Boost Converters, Sepehr Saadatmand, Pourya Shamsi, Mehdi Ferdowsi

Electrical and Computer Engineering Faculty Research & Creative Works

In this study, a heuristic dynamic programming controller is proposed to control a boost converter. Conventional controllers such as proportional-integral-derivative (PID) or proportional-integral (PI) are designed based on the linearized small-signal model near the operating point. Therefore, the performance of the controller during the start-up, the load change, or the input voltage variation is not optimal since the system model changes by varying the operating point. The heuristic dynamic programming controller optimally controls the boost converter by following the approximate dynamic programming. The advantage of the HDP is that the neural network-based characteristic of the proposed controller enables boost converters …

Optimal Boundary Detection Using Autonomous Mobile Sensors, Phillip Justin Kearns

Dissertations and Theses

A fundamental challenge to modern science and engineering is the ability to rapidly and accurately determine the spatial extent of environmental phenomena. In monitoring the spread of hazardous pollution, for example, all points with pollutant concentration above or below a fixed threshold can be considered as two classes in a binary classification problem. In this instance, the goal is to accurately estimate the decision boundary as quickly as possible. To generate models and predictions, scientists must choose their sampling locations from a vast array of possibilities. This thesis develops a policy for determining the optimal sample locations for a fixed …

The Voltage Regulation Of Boost Converters Using Dual Heuristic Programming, Sepehr Saadatmand, Sima Azizi, Mohammadamir Kavousi

Graduate Student Research & Creative Works

In this paper, a dual heuristic programming controller is proposed to control a boost converter. Conventional controllers such as proportional-integral-derivative (PID) or proportional-integral (PI) are designed based on the linearized small-signal model near the operating point. Therefore, the performance of the controller during start-up, load change, or input voltage variation is not optimal since the system model changes by varying the operating point. The dual heuristic programming controller optimally controls the boost converter by following the approximate dynamic programming. The advantage of the DHP is that the neural network–based characteristic of the proposed controller enables boost converters to easily cope …

Deployment Of Deep Reinforcement Learning Algorithms On Physical Robotic Systems, P. Surya .

Manipal Institute of Technology, Manipal Theses and Dissertations

In the recent years, Reinforcement Learning (RL), has surfaced as an impressive tool for solving complex sequential decision-making problems in control theory. Its major grasp on control theory has been on Markov decision problems (MDP) and its variants. It overcomes the long observed problems of the curse of dimensionality and the curse of modeling. This outperforms the classical Dynamic Programming (DP) that falter under these curses. The work is to use this advantage in our favour to leverage autonomous navigation in a model learnable environment or otherwise. The work involves the use of a Meta OS called Robot Operating System …

Satellite Constellation Deployment And Management, Joseph Ryan Kopacz

Electronic Theses and Dissertations

This paper will review results and discuss a new method to address the deployment and management of a satellite constellation. The first two chapters will explorer the use of small satellites, and some of the advances in technology that have enabled small spacecraft to maintain modern performance requirements in incredibly small packages.

The third chapter will address the multiple-objective optimization problem for a global persistent coverage constellation of communications spacecraft in Low Earth Orbit. A genetic algorithm was implemented in MATLAB to explore the design space – 288 trillion possibilities – utilizing the Satellite Tool Kit (STK) software developers kit. …

Deep Reinforcement Learning For The Optimization Of Building Energy Control And Management, Jun Hao

Electronic Theses and Dissertations

Most of the current game-theoretic demand-side management methods focus primarily on the scheduling of home appliances, and the related numerical experiments are analyzed under various scenarios to achieve the corresponding Nash-equilibrium (NE) and optimal results. However, not much work is conducted for academic or commercial buildings. The methods for optimizing academic-buildings are distinct from the optimal methods for home appliances. In my study, we address a novel methodology to control the operation of heating, ventilation, and air conditioning system (HVAC).

We assume that each building in our campus is equipped with smart meter and communication system which is envisioned in …

Uav Navigation, Tracking, And Interception Using Deep Reinforcement Learning, Ali Alhaj Darwish

All ETDs from UAB

We propose a self-supervised, model-free deep reinforcement learning architecture with cascade reward for unmanned aerial vehicle navigation and target interception in a 3D environment. The first contribution of the dissertation solves the problem of partial observability when non-linear function approximators are used for learning stochastic policies. The second contribution optimizes the problem of maximizing the total expected rewards. The third contribution trains the agent in a photo-realistic environment with a real physics engine. To achieve these goals, a deep Q-network that combines double and dueling architectures is adopted as a value function approximator, and the prioritized experience replays the sample …

Wind Power Forecasting Methods Based On Deep Learning: A Survey, Xing Deng, Haijian Shao, Chunlong Hu, Dengbiao Jiang, Yingtao Jiang

Electrical & Computer Engineering Faculty Research

Accurate wind power forecasting in wind farm can effectively reduce the enormous impact on grid operation safety when high permeability intermittent power supply is connected to the power grid. Aiming to provide reference strategies for relevant researchers as well as practical applications, this paper attempts to provide the literature investigation and methods analysis of deep learning, enforcement learning and transfer learning in wind speed and wind power forecasting modeling. Usually, wind speed and wind power forecasting around a wind farm requires the calculation of the next moment of the definite state, which is usually achieved based on the state of …