Computer Sciences Commons^™

Learning Transferable Meta-Policies For Hierarchical Task Decomposition And Planning Composition, Predrag Djurdjevic

Computer Science and Engineering Dissertations - Archive

In real world scenarios where situated agents are faced with dynamic, high-dimensional, partially observable environments with action and reward uncertainty, the traditional states space Reinforcement Learning (RL) becomes easily prohibitively large for policy learning. In such scenarios, addressing the curse of dimensionality and eventual transfer to closely related tasks is one of the principal challenges and motivations for Hierarchical Reinforcement Learning (HRL). The prime appeal of hierarchal and particularly recursive approaches is in effective factored state, transition and reward representations which abstract out aspects that are not relevant to subtasks and allow potential transfer of skills which represent solutions to …

Hierarchical Multiagent Reinforcement Learning For Maritime Traffic Management, Arambam James Singh, Akshat Kumar, Hoong Chuin Lau

Research Collection School Of Computing and Information Systems

Increasing global maritime traffic coupled with rapid digitization and automation in shipping mandate developing next generation maritime traffic management systems to mitigate congestion, increase safety of navigation, and avoid collisions in busy and geographically constrained ports (such as Singapore's). To achieve these objectives, we model the maritime traffic as a large multiagent system with individual vessels as agents, and VTS (Vessel Traffic Service) authority as a regulatory agent. We develop a hierarchical reinforcement learning approach where vessels first select a high level action based on the underlying traffic flow, and then select the low level action that determines their future …

Using Logical Specifications For Multi-Objective Reinforcement Learning, Kolby Nottingham

Undergraduate Honors Theses

In the multi-objective reinforcement learning (MORL) paradigm, the relative importance of environment objectives is often unknown prior to training, so agents must learn to specialize their behavior to optimize different combinations of environment objectives that are specified post-training. These are typically linear combinations, so the agent is effectively parameterized by a weight vector that describes how to balance competing environment objectives. However, we show that behaviors can be successfully specified and learned by much more expressive non-linear logical specifications. We test our agent in several environments with various objectives and show that it can generalize to many never-before-seen specifications.

Using Reinforcement Learning To Minimize The Probability Of Delay Occurrence In Transportation, Zhiguang Cao, Hongliang Guo, Wen Song, Kaizhou Gao, Zhengghua Chen, Le Zhang, Xuexi Zhang

Research Collection School Of Computing and Information Systems

Reducing traffic delay is of crucial importance for the development of sustainable transportation systems, which is a challenging task in the studies of stochastic shortest path (SSP) problem. Existing methods based on the probability tail model to solve the SSP problem, seek for the path that minimizes the probability of delay occurrence, which is equal to maximizing the probability of reaching the destination before a deadline (i.e., arriving on time). However, they suffer from low accuracy or high computational cost. Therefore, we design a novel and practical Q-learning approach where the converged Q-values have the practical meaning as the actual …

Deep Reinforcement Learning For The Optimization Of Building Energy Control And Management, Jun Hao

Electronic Theses and Dissertations

Most of the current game-theoretic demand-side management methods focus primarily on the scheduling of home appliances, and the related numerical experiments are analyzed under various scenarios to achieve the corresponding Nash-equilibrium (NE) and optimal results. However, not much work is conducted for academic or commercial buildings. The methods for optimizing academic-buildings are distinct from the optimal methods for home appliances. In my study, we address a novel methodology to control the operation of heating, ventilation, and air conditioning system (HVAC).

We assume that each building in our campus is equipped with smart meter and communication system which is envisioned in …

Wind Power Forecasting Methods Based On Deep Learning: A Survey, Xing Deng, Haijian Shao, Chunlong Hu, Dengbiao Jiang, Yingtao Jiang

Electrical & Computer Engineering Faculty Research

Accurate wind power forecasting in wind farm can effectively reduce the enormous impact on grid operation safety when high permeability intermittent power supply is connected to the power grid. Aiming to provide reference strategies for relevant researchers as well as practical applications, this paper attempts to provide the literature investigation and methods analysis of deep learning, enforcement learning and transfer learning in wind speed and wind power forecasting modeling. Usually, wind speed and wind power forecasting around a wind farm requires the calculation of the next moment of the definite state, which is usually achieved based on the state of …

Satellite Constellation Deployment And Management, Joseph Ryan Kopacz

Electronic Theses and Dissertations

This paper will review results and discuss a new method to address the deployment and management of a satellite constellation. The first two chapters will explorer the use of small satellites, and some of the advances in technology that have enabled small spacecraft to maintain modern performance requirements in incredibly small packages.

The third chapter will address the multiple-objective optimization problem for a global persistent coverage constellation of communications spacecraft in Low Earth Orbit. A genetic algorithm was implemented in MATLAB to explore the design space – 288 trillion possibilities – utilizing the Satellite Tool Kit (STK) software developers kit. …

Robot Arm Control Method Based On Deep Reinforcement Learning, Heyu Li, Zhilong Zhao, Gu Lei, Liqin Guo, Zeng Bi, Tingyu Lin

Journal of System Simulation

Abstract: Deep reinforcement learning continues to explore in the environment and adjusts the neural network parameters by the reward function. The actual production line can not be used as the trial and error environment for the algorithm, so there is not enough data. For that, this paper constructs a virtual robot arm simulation environment, including the robot arm and the object. The Deep Deterministic Policy Gradient (DDPG),in which the state variables and reward function are set,is trained by deep reinforcement learning algorithm in the simulation environment to realize the target of controlling the robot arm to move the gripper below …

Learning Robot Manipulation Tasks Via Observation, Michail Theofanidis

Computer Science and Engineering Dissertations - Archive

The coexistence of humans and robots has been the aspiration of many scientific endeavors in the past century. Most anthropomorphic or industrial robots are highly articulated and complex machines, which are designed to carry out tasks that often involve the manipulation of physical objects. Traditionally, robots learn how to perform such tasks with the aid of a human programmer or operator. In this regard, the human acts as a teacher who provides a demonstration of a task. From the data of the demonstration, the robot must learn a state-action mapping that accomplishes the task. This state-action mapping is often addressed …

Toward Self-Reconfigurable Parametric Systems: Reinforcement Learning Approach, Ting-Yu Mu

Dissertations

For the ongoing advancement of the fields of Information Technology (IT) and Computer Science, machine learning-based approaches are utilized in different ways in order to solve the problems that belong to the Nondeterministic Polynomial time (NP)-hard complexity class or to approximate the problems if there is no known efficient way to find a solution. Problems that determine the proper set of reconfigurable parameters of parametric systems to obtain the near optimal performance are typically classified as NP-hard problems with no efficient mathematical models to obtain the best solutions. This body of work aims to advance the knowledge of machine learning …

Domain Adaptation In Unmanned Aerial Vehicles Landing Using Reinforcement Learning, Pedro Lucas Franca Albuquerque

School of Computing: Dissertations, Theses, and Student Research

Landing an unmanned aerial vehicle (UAV) on a moving platform is a challenging task that often requires exact models of the UAV dynamics, platform characteristics, and environmental conditions. In this thesis, we present and investigate three different machine learning approaches with varying levels of domain knowledge: dynamics randomization, universal policy with system identification, and reinforcement learning with no parameter variation. We first train the policies in simulation, then perform experiments both in simulation, making variations of the system dynamics with wind and friction coefficient, then perform experiments in a real robot system with wind variation. We initially expected that providing …

Comparison Of Rl Algorithms For Learning To Learn Problems, Adolfo Gonzalez Iii

Theses and Dissertations

Machine learning has been applied to many different problems successfully due to the expressiveness of neural networks and simplicity of first order optimization algorithms. The latter being a vital piece needed for training large neural networks efficiently. Many of these algorithms were produced with behavior produced by experiments and intuition. An interesting question that comes to mind is that rather than observing and then designing algorithms with beneficial behaviors, can these algorithms be learned through a reinforcement learning by modeling optimization as a game. This paper explores several reinforcement learning algorithms which are applied to learn policies suited for optimization.

Multi-Agent Collaborative Exploration Through Graph-Based Deep Reinforcement Learning, Tianze Luo, Budhitama Subagdja, Ah-Hwee Tan, Ah-Hwee Tan

Research Collection School Of Computing and Information Systems

Autonomous exploration by a single or multiple agents in an unknown environment leads to various applications in automation, such as cleaning, search and rescue, etc. Traditional methods normally take frontier locations and segmented regions of the environment into account to efficiently allocate target locations to different agents to visit. They may employ ad hoc solutions to allocate the task to the agents, but the allocation may not be efficient. In the literature, few studies focused on enhancing the traditional methods by applying machine learning models for agent performance improvement. In this paper, we propose a graph-based deep reinforcement learning approach …

Deep Reinforcement Learning-Based Portfolio Management, Nitin Kanwar

Computer Science and Engineering Theses - Archive

Machine Learning is at the forefront of every field today. The subfields of Machine Learning called Reinforcement Learning and Deep Learning, when combined have given rise to advanced algorithms which have been successful at reaching or surpassing the human-level performance at playing Atari games to defeating multiple times champion at Go. These successes of Machine Learning have attracted the interest of the financial community and have raised the question if these techniques could also be applied in detecting patterns in the financial markets. Until recently, mathematical formulations of dynamical systems in the context of Signal Processing and Control Theory have …

Learning Representations Using Reinforcement Learning, Sourabh Bose

Computer Science and Engineering Dissertations - Archive

The framework of reinforcement learning is a powerful suite of algorithms that can learn generalized solutions to complex decision making problems. However, the applications of reinforcement learning algorithms to traditional machine learning problems such as clustering, classification and representation learning, have rarely been explored. With the advent of large amounts of data, robust models are required which can extract meaningful representations from the data that can potentially be applied to new unseen tasks. The presented work investigates the applications of reinforcement learning algorithms in the perspective of transfer learning by applying algorithms in the framework of reinforcement learning to address …

Dp-Q(Λ): Real-Time Path Planning For Multi-Agent In Large-Scale Web3d Scene, Fengting Yan, Jinyuan Jia

Journal of System Simulation

Abstract: The path planning of multi-agent in an unknown large-scale scene needs an efficient and stable algorithm, and needs to solve multi-agent collision avoidance problem, and then completes a real-time path planning in Web3D. To solve above problems, the DP-Q(λ) algorithm is proposed; and the direction constraints, high reward or punishment weight training methods are used to adjust the values of reward or punishment by using a probability p (0-1 random number). The value from reward or punishment determines its next step path planning strategy. If the next position is free, the agent could walk to it. The above strategy …

Application Of Retrograde Analysis To Fighting Games, Kristen Yu

Electronic Theses and Dissertations

With the advent of the fighting game AI competition, there has been recent interest in two-player fighting games. Monte-Carlo Tree-Search approaches currently dominate the competition, but it is unclear if this is the best approach for all fighting games. In this thesis we study the design of two-player fighting games and the consequences of the game design on the types of AI that should be used for playing the game, as well as formally define the state space that fighting games are based on. Additionally, we also characterize how AI can solve the game given a simultaneous action game model, …

A Deep Recurrent Q Network Towards Self-Adapting Distributed Microservice Architecture, Basel Magableh, Muder Almiani

Articles

One desired aspect of microservice architecture is the ability to self-adapt its own architecture and behavior in response to changes in the operational environment. To achieve the desired high levels of self-adaptability, this research implements distributed microservice architecture model running a swarm cluster, as informed by the Monitor, Analyze, Plan, and Execute over a shared Knowledge (MAPE-K) model. The proposed architecture employs multiadaptation agents supported by a centralized controller, which can observe the environment and execute a suitable adaptation action. The adaptation planning is managed by a deep recurrent Q-learning network (DRQN). It is argued that such integration between DRQN …

Bridging Act-R And Project Malmo, Developing Models Of Behavior In Complex Environments, David M. Schwartz

Honors Theses

Cognitive architectures such as ACT-R provide a system for simulating the mind and human behavior. On their own they model decision making of an isolated agent. However, applying a cognitive architecture to a complex environment yields more interesting results about how people make decisions in more realistic scenarios. Furthermore, cognitive architectures enable researchers to study human behavior in dangerous tasks which cannot be tested because they would harm participants. Nonetheless, these architectures aren’t commonly applied to such environments as they don’t come with one. It is left to the researcher to develop a task environment for their model. The difficulty …

Using Reinforcement Learning In A Simulated Intelligent Tutoring System, Manohar Sai Jasti

Graduate Research Theses & Dissertations

I used reinforcement learning to investigate which categories of hints are most efficient in an intelligent tutoring system for human anatomy. Efficiency is defined as minimizing the time it takes the student to learn the material. When a student gives a wrong answer, the tutor can give them a text hint, a diagrammatic hint, or a video clip. Each type of hint takes a different amount of time to deliver and takes the student a different amount of time to understand.

I built a simulator for the intelligent tutoring system to collect data from simulated students. I implemented reinforcement learning, …

Regression Tree Construction For Reinforcement Learning Problems With A General Action Space, Anthony S. Bush Jr

College of Graduate Studies: Theses & Dissertations

Part of the implementation of Reinforcement Learning is constructing a regression of values against states and actions and using that regression model to optimize over actions for a given state. One such common regression technique is that of a decision tree; or in the case of continuous input, a regression tree. In such a case, we fix the states and optimize over actions; however, standard regression trees do not easily optimize over a subset of the input variables\cite{Card1993}. The technique we propose in this thesis is a hybrid of regression trees and kernel regression. First, a regression tree splits over …

Off-Policy Integral Reinforcement Learning For Semi-Global Constrained Output Regulation Of Continuous-Time Linear Systems, Yongliang Yang, Xianzhong Chen, Yixin Yin, Donald C. Wunsch

Electrical and Computer Engineering Faculty Research & Creative Works

This paper presents a data-driven method based on off-policy integral reinforcement learning to solve the semi-global output regulation of continuous-time linear systems with input saturation. A family of state feedback laws for the input constrained output regulation problem is designed based on solving an algebraic Riccati equation. In contrast to the existing methods, complete knowledge of the system dynamics is no longer required in this paper. Instead, the data collected from online implementation is efficiently utilized to design the controller. Therefore, the controller design in this paper is data driven. It is shown that the presented method can find feedback …

Adopt: Combining Parameter Tuning And Adaptive Operator Ordering For Solving A Class Of Orienteering Problems, Aldy Gunawan, Hoong Chuin Lau, Kun Lu

Research Collection School Of Computing and Information Systems

Two fundamental challenges in local search based metaheuristics are how to determine parameter configurations and design the underlying Local Search (LS) procedure. In this paper, we propose a framework in order to handle both challenges, called ADaptive OPeraTor Ordering (ADOPT). In this paper, The ADOPT framework is applied to two metaheuristics, namely Iterated Local Search (ILS) and a hybridization of Simulated Annealing and ILS (SAILS) for solving two variants of the Orienteering Problem: the Team Dependent Orienteering Problem (TDOP) and the Team Orienteering Problem with Time Windows (TOPTW). This framework consists of two main processes. The Design of Experiment (DOE) …

Probabilistic Guided Exploration For Reinforcement Learning In Self-Organizing Neural Networks, Peng Wang, Weigui Jair Zhou, Di Wang, Ah-Hwee Tan

Research Collection School Of Computing and Information Systems

Exploration is essential in reinforcement learning, which expands the search space of potential solutions to a given problem for performance evaluations. Specifically, carefully designed exploration strategy may help the agent learn faster by taking the advantage of what it has learned previously. However, many reinforcement learning mechanisms still adopt simple exploration strategies, which select actions in a pure random manner among all the feasible actions. In this paper, we propose novel mechanisms to improve the existing knowledgebased exploration strategy based on a probabilistic guided approach to select actions. We conduct extensive experiments in a Minefield navigation simulator and the results …

Bounding Box Improvement With Reinforcement Learning, Andrew Lewis Cleland

Dissertations and Theses

In this thesis, I explore a reinforcement learning technique for improving bounding box localizations of objects in images. The model takes as input a bounding box already known to overlap an object and aims to improve the fit of the box through a series of transformations that shift the location of the box by translation, or change its size or aspect ratio. Over the course of these actions, the model adapts to new information extracted from the image. This active localization approach contrasts with existing bounding-box regression methods, which extract information from the image only once. I implement, train, and …

Towards A Physio-Cognitive Model Of The Exploration Exploitation Trade-Off., David M. Schwartz, Christopher L. Dancy

Faculty Conference Papers and Presentations

Managing the exploration vs exploitation trade-off is an important part of our everyday lives. It occurs in minor decisions such as choosing what music to listen to as well as major decisions, such as picking a research direction to pursue. The dilemma is the same despite the context: does one exploit the environment, using current knowledge to acquire a satisfactory solution, or explore other options and potentially find a better answer. An accurate cognitive model must be able to handle this trade-off because of the importance it plays in our lives. We are developing physio-cognitive models to better understand how …

Pseudorehearsal In Actor-Critic Agents With Neural Network Function Approximation, Vladimir Marochko, Leonard Johard, Manuel Mazzara, Luca Longo

Articles

Catastrophic forgetting has a significant negative impact in reinforcement learning. The purpose of this study is to investigate how pseudorehearsal can change performance of an actor-critic agent with neural-network function approximation. We tested agent in a pole balancing task and compared different pseudorehearsal approaches. We have found that pseudorehearsal can assist learning and decrease forgetting.

Intelligent And Secure Underwater Acoustic Communication Networks, Chaofeng Wang

Dissertations, Master's Theses and Master's Reports

Underwater acoustic (UWA) communication networks are promising techniques for medium- to long-range wireless information transfer in aquatic applications. The harsh and dynamic water environment poses grand challenges to the design of UWA networks. This dissertation leverages the advances in machine learning and signal processing to develop intelligent and secure UWA communication networks. Three research topics are studied: 1) reinforcement learning (RL)-based adaptive transmission in UWA channels; 2) reinforcement learning-based adaptive trajectory planning for autonomous underwater vehicles (AUVs) in under-ice environments; 3) signal alignment to secure underwater coordinated multipoint (CoMP) transmissions.

First, a RL-based algorithm is developed for adaptive transmission in …

An Integrated Framework For Modeling And Predicting Spatiotemporal Phenomena In Urban Environments, Tuc Viet Le

Dissertations and Theses Collection (Open Access)

This thesis proposes a general solution framework that integrates methods in machine learning in creative ways to solve a diverse set of problems arising in urban environments. It particularly focuses on modeling spatiotemporal data for the purpose of predicting urban phenomena. Concretely, the framework is applied to solve three specific real-world problems: human mobility prediction, trac speed prediction and incident prediction. For human mobility prediction, I use visitor trajectories collected a large theme park in Singapore as a simplified microcosm of an urban area. A trajectory is an ordered sequence of attraction visits and corresponding timestamps produced by a visitor. …

Modeling Trajectories With Recurrent Neural Networks, Hao Wu, Ziyang Chen, Weiwei Sun, Baihua Zheng, Wei Wang

Research Collection School Of Computing and Information Systems

Modeling trajectory data is a building block for many smart-mobility initiatives. Existing approaches apply shallow models such as Markov chain and inverse reinforcement learning to model trajectories, which cannot capture the long-term dependencies. On the other hand, deep models such as Recurrent Neura lNetwork (RNN) have demonstrated their strength of modeling variable length sequences. However, directly adopting RNN to model trajectories is not appropriate because of the unique topological constraints faced by trajectories. Motivated by these findings, we design two RNN-based models which can make full advantage of the strength of RNN to capture variable length sequence and meanwhile to …