Artificial Intelligence and Robotics Commons^™

Advancing Household Robotics: Deep Interactive Reinforcement Learning For Efficient Training And Enhanced Performance, Arpita Soni, Sujatha Alla, Suresh Dodda, Hemanth Volikatla

Engineering Management & Systems Engineering Faculty Publications

The market for domestic robots—made to perform household chore, is growing as these robots relieve people of everyday responsibilities. Domestic robots are generally welcomed for their role in easing human labour, in contrast to industrial robots, which are frequently criticised for displacing human workers. But before these robots can carry out domestic chores, they need to become proficient in a number of minor activities, such as recognizing their surroundings, making decisions, and picking up on human behaviours. Reinforcement learning, or RL, has emerged as a key robotics technology that enables robots to interact with their environment and learn how to …

Task Scheduling For Internet Of Vehicles Based On Deep Reinforcement Learning In Edge Computing, Xiang Ju, Shengchao Su, Chaojie Xu, Beibei He

Journal of System Simulation

Abstract: Aiming at the offloading and execution of delay-constrained computing tasks for internet of vehicles in edge computing, a task scheduling method based on deep reinforcement learning is proposed. In multi-edge server scenario, a software-defined network-aided internet of vehicles task offloading system is built. On this basis, the task scheduling model of vehicle computation offloading is given. According to the characteristics of task scheduling, a scheduling method based on an improved pointer network is designed. Considering the complexity of task scheduling and computing resource allocation, the deep reinforcement learning algorithm is used to train the pointer network. The vehicle offloading …

Marss: Multi-Agent Reinforcement Learning For Satellite Swarms, Nicholas J. Yielding

Theses and Dissertations

Multi-agent systems and swarms in spacecraft formation flying are of ever-increasing importance in a contested space environment—use of multiple spacecraft to contribute to a cooperative mission potentially increases positive outcomes on orbit, while autonomy becomes an ever more important requirement to reduce reaction time in dynamic situations and lower the burden on space operators. This research explores difficult swarm Guidance Navigation and Control (GNC) scenarios using Deep Reinforcement Learning (DRL). DRL polices are trained to provide guidance inputs to agents in multi-agent swarm environments for completing complex, teamwork focused objectives in geosynchronous orbit. An example scenario is explored for a …

Advances In Quaternion-Valued Neural Networks, Jeremiah P. Bill

Theses and Dissertations

This dissertation investigates the construction, optimization, and application of quaternion neural networks (QNNs) to Department of Defense (DoD) related problem sets. QNNs are a type of neural network wherein the weights, biases, and input values are all represented as quaternion numbers. This work provides a critical evaluation of the myriad different quaternion backpropagation derivations that exist in the literature, testing the performance of each on a range of regression problem sets. The optimization dynamics of QNNs are explored, presenting visualizations of QNN loss surfaces and a novel method for assessing the “smoothness” of these loss surfaces. Finally, this dissertation presents …

Obstacle Avoidance Path Planning And Simulation Of Mobile Picking Robot Based On Dppo, Junqiang Lin, Hongjun Wang, Xiangjun Zou, Po Zhang, Chengen Li, Yipeng Zhou, Shujie Yao

Journal of System Simulation

Abstract: Aiming at the autonomous decision-making difficulty of mobile picking robots in random and changeable complicated path environment during field operations, an autonomous obstacle avoidance path planning method based on deep reinforcement learning is propose. By setting the state space and action space and using the artificial potential field method to design the reward function, an obstacle penalty coefficient setting method based on collision cone collision avoidance detection is proposed to improve the autonomous collision avoidance ability. A virtual simulation system is constructed, in which the learning and training of the mobile picking robot is carried out and verified by …

Intelligent Path Planning For Mobile Robots Based On Sac Algorithm, Laiyi Yang, Jing Bi, Haitao Yuan

Journal of System Simulation

Abstract: Aiming at the high dimension, slow convergence and complex modelling of traditional path planning algorithms for mobile robots, a new intelligent path planning algorithm is proposed, which is based on deep reinforcement learning soft actor-critic (SAC) algorithm to save the poor performance of robot in complicated environments with static and dynamic obstacles. An improved reward function is designed to enable mobile robots to quickly avoid obstacles and reach targets by using state dynamic normalization and priority experience pool techniques. To evaluate the performance, a pygame-based simulation environment is constructed. Compared with proximal policy optimization(PPO) algorithm, experimental …

Intelligent Air Defense Task Assignment Based On Assignment Strategy Optimization Algorithm, Jiayi Liu, Gang Wang, Qiang Fu, Xiangke Guo, Siyuan Wang

Journal of System Simulation

Abstract: Aiming at the insufficient solving speed of assignment strategy optimization algorithm in largescale scenarios, deep reinforcement learning is combined with Markov decision process to carry out the intelligent large-scale air defense task assignment. According to the characteristics of large-scale air defense operations, Markov decision process is used to model the agent and a digital battlefield simulation environment is built. Air defense task assignment agent is designed and trained in digital battlefield simulation environment through proximal policy optimization algorithm. The feasibility and advantage of the method are verified by taking a large-scale ground-to-air countermeasure mission as an example.

Generalization Through Diversity: Improving Unsupervised Environment Design, Wenjun Li, Pradeep Varakantham, Dexun Li

Research Collection School Of Computing and Information Systems

Agent decision making using Reinforcement Learning (RL) heavily relies on either a model or simulator of the environment (e.g., moving in an 8x8 maze with three rooms, playing Chess on an 8x8 board). Due to this dependence, small changes in the environment (e.g., positions of obstacles in the maze, size of the board) can severely affect the effectiveness of the policy learned by the agent. To that end, existing work has proposed training RL agents on an adaptive curriculum of environments (generated automatically) to improve performance on out-of-distribution (OOD) test scenarios. Specifically, existing research has employed the potential for the …

Joint Optimization Strategy Of Computing Offloading And Edge Caching For Intelligent Connected Vehicles, Fei Ding, Yuchen Sha, Ying Hong, Xiao Kuai, Dengyin Zhang

Journal of System Simulation

To guarantee the low-delay communication of intelligent connected vehicles, the V2X channel model and the multi-access edge computing (MEC) technology, are used to carry out the research of the joint optimization strategy of computing offloading and edge caching.An intelligent connected vehicle with task offloading and edge caching model least-deep deterministic policy gradient(L-DDPG) is developed.By integrating the vehicular local and edge computing resources, the classification processing of different computing tasks in V2X scenarios is supported.The vehicular computing request is prejudged by edge platform to ensure the rapid response of continuous homogeneous computing tasks. Combining with the least recently …

Deep Reinforcement Learning-Based Control Strategy For Boost Converter, Yuxuan Dai, Chenggang Cui

Journal of System Simulation

Abstract: In view of the problems of model uncertainty and nonlinearity in bus voltage stability control of Boost converter, an intelligent control strategy based on model-free deep reinforcement learning(RL) is proposed. RL double DQN(DDQN) algorithm and deep deterministic policy gradient(DDPG) algorithm are used, and the Boost converter controller is designed. The state, action space, reward function, and neural network are also designed to improve the dynamic performance of the controller. The joint simulation of the Boost converter model and RL agent is realized by RL modelica(RLM …

Research Progress Of Opponent Modeling Based On Deep Reinforcement Learning, Haotian Xu, Long Qin, Junjie Zeng, Yue Hu, Qi Zhang

Journal of System Simulation

Abstract: Deep reinforcement learning is an agent modeling method with both deep learning feature extraction ability and reinforcement learning sequence decision-making ability, which can make up for the depleted non-stationary adaptation, complex feature selection and insufficient state-space representation ability of traditional opponent modeling. The deep reinforcement learning-based opponent modeling methods are divided into two categories, explicit modeling and implicit modeling, and the corresponding theories, models, algorithms and applicable scenarios are sorted out according to the categories. The applications of deep reinforcement learning-based opponent modeling techniques on different fields are introduced. The key problems and future development are summarized to provide …

A Review On Learning To Solve Combinatorial Optimisation Problems In Manufacturing, Cong Zhang, Yaoxin Wu, Yining Ma, Wen Song, Zhang Le, Zhiguang Cao, Jie Zhang

Research Collection School Of Computing and Information Systems

An efficient manufacturing system is key to maintaining a healthy economy today. With the rapid development of science and technology and the progress of human society, the modern manufacturing system is becoming increasingly complex, posing new challenges to both academia and industry. Ever since the beginning of industrialisation, leaps in manufacturing technology have always accompanied technological breakthroughs from other fields, for example, mechanics, physics, and computational science. Recently, machine learning (ML) technology, one of the crucial subjects of artificial intelligence, has made remarkable progress in many areas. This study thoroughly reviews how ML, specifically deep (reinforcement) learning, motivates new ideas …

Sc-Matrl: Semi-Centralized Multi-Agent Transfer Reinforcement Learning, Ayesha Siddika Nipu

Graduate Theses/Dissertations

Distributed decision-making in multi-agent systems (MAS) poses significant challenges for interactive behavior learning in both cooperative and competitive environments. While reinforcement learning (RL) has shown great success in single-agent domains like Checkers, Chess and Go, researchers are motivated to extend RL to MAS. However, as the number of agents increases, effectively dealing with each agent becomes increasingly complex. To mitigate the resulting complexity, a semi-centralized Multi-Agent Influence Dense Reinforcement Learning (MAIDRL) algorithm was previously developed, enhancing agent influence maps to facilitate effective multi-agent control in StarCraft Multi-Agent Challenge (SMAC) scenarios. While MAIDRL shows improved performance in homogeneous multi-agent scenarios, it …

Pruning The Communication Bandwidth Between Reinforcement Learning Agents Through Causal Inference: An Innovative Approach To Designing A Smart Grid Power System, Xianjie Zhang, Yu Liu, Wenjun Li, Chen Gong

Research Collection School Of Computing and Information Systems

Electricity demands are increasing significantly and the traditional power grid system is facing huge challenges. As the desired next-generation power grid system, smart grid can provide secure and reliable power generation, and consumption, and can also realize the system’s coordinated and intelligent power distribution. Coordinating grid power distribution usually requires mutual communication between power distributors to accomplish coordination. However, the power network is complex, the network nodes are far apart, and the communication bandwidth is often expensive. Therefore, how to reduce the communication bandwidth in the cooperative power distribution process task is crucially important. One way to tackle this problem …

Pruning The Communication Bandwidth Between Reinforcement Learning Agents Through Causal Inference: An Innovative Approach To Designing A Smart Grid Power System, Xianjie Zhang, Yu Liu, Wenjun Li, Chen Gong

Research Collection School Of Computing and Information Systems

Electricity demands are increasing significantly and the traditional power grid system isfacing huge challenges. As the desired next-generation power grid system, smart grid can providesecure and reliable power generation, and consumption, and can also realize the system’s coordinatedand intelligent power distribution. Coordinating grid power distribution usually requiresmutual communication between power distributors to accomplish coordination. However, the powernetwork is complex, the network nodes are far apart, and the communication bandwidth is oftenexpensive. Therefore, how to reduce the communication bandwidth in the cooperative power distributionprocess task is crucially important. One way to tackle this problem is to build mechanismsto selectively send out …

Research On Fire Emergency Evacuation Simulation Based On Cooperative Deep Reinforcement Learning, Lingjia Ni, Xiaoxia Huang, Hongga Li, Zibo Zhang

Journal of System Simulation

Abstract: The fire accident is a major threat to the public safety, in which the high temperature, toxic and harmful gases seriously interfer the selection of the evacuation routes. Deep reinforcement learning is introduced into the research of emergency evacuation simulation, and a cooperative double deep Q network algorithm is proposed for the multi-agent environment. A fire scene model that changes dynamically over time is established to provide the real-time information on the distribution of the dangerous areas for the evacuation. The independent agent neural networks are integrated and the multi-agent unified deep neural network is established to realize the …

A Self-Learning Intersection Control System For Connected And Automated Vehicles, Ardeshir Mirbakhsh

Dissertations

This study proposes a Decentralized Sparse Coordination Learning System (DSCLS) based on Deep Reinforcement Learning (DRL) to control intersections under the Connected and Automated Vehicles (CAVs) environment. In this approach, roadway sections are divided into small areas; vehicles try to reserve their desired area ahead of time, based on having a common desired area with other CAVs; the vehicles would be in an independent or coordinated state. Individual CAVs are set accountable for decision-making at each step in both coordinated and independent states. In the training process, CAVs learn to minimize the overall delay at the intersection. Due to the …

Pervasive Machine Learning For Smart Radio Environments Enabled By Reconfigurable Intelligent Surfaces, George C. Alexandropoulos, Kyriakos Stylianopoulos, Chongwen Huang, Chau Yuen, Mehdi Bennis, Mérouane Debbah

Machine Learning Faculty Publications

The emerging technology of Reconfigurable Intelligent Surfaces (RISs) is provisioned as an enabler of smart wireless environments, offering a highly scalable, low-cost, hardware-efficient, and almost energy-neutral solution for dynamic control of the propagation of electromagnetic signals over the wireless medium, ultimately providing increased environmental intelligence for diverse operation objectives. One of the major challenges with the envisioned dense deployment of RISs in such reconfigurable radio environments is the efficient configuration of multiple metasurfaces with limited, or even the absence of, computing hardware. In this paper, we consider multi-user and multi-RIS-empowered wireless systems, and present a thorough survey of the online …

A Devs-Based Formal Description Method For Complex Product Behavior Models, Qingquan Lin, Jiaran Yang, Heming Zhang

Journal of System Simulation

Abstract: For the online optimization of pedestrian flow control in subway station, an algorithm frame for pedestrian flow control in subway station based on machine learning is designed. The pedestrian flow control process of a subway station during morning rush hour is selected,and the agent-based model is built to simulate the control process. The training data is collected through the multiple runs of the model, which is used as the input of deep reinforcement learning network, and the mature net is obtained through adequate training to provide the optimizing scheduling policy. Linking the actual data with the mature net …

Research On Optimization Of Airport Cargo Business Based On Deep Reinforcement Learning, Hongwei Wang, Peng Yang

Journal of System Simulation

Abstract: An intelligent agent technology architecture is adopted to the simulation model development of airport cargo business. Aiming at the optimization of airport cargo resources, a decision support system framework combining deep reinforcement learning (DRL) and airport cargo business simulation model is proposed. The simulated results are applied as the training data of the DRL network, and the DRL is used to optimize operation parameter of the simulation model. The mature system can be run online, which can provide optimized operation order in real time. In order to verify the effectiveness of the architecture, model development and experiments are conducted …

Heterogeneous Attentions For Solving Pickup And Delivery Problem Via Deep Reinforcement Learning, Jingwen Li, Liang Xin, Zhiguang Cao, Andrew Lim, Wen Song, Jie Zhang

Research Collection School Of Computing and Information Systems

Recently, there is an emerging trend to apply deep reinforcement learning to solve the vehicle routing problem (VRP), where a learnt policy governs the selection of next node for visiting. However, existing methods could not handle well the pairing and precedence relationships in the pickup and delivery problem (PDP), which is a representative variant of VRP. To address this challenging issue, we leverage a novel neural network integrated with a heterogeneous attention mechanism to empower the policy in deep reinforcement learning to automatically select the nodes. In particular, the heterogeneous attention mechanism specifically prescribes attentions for each role of the …

Job Scheduling And Simulation In Cloud Based On Deep Reinforcement Learning, Qirui Li, Xinyi Peng

Journal of System Simulation

Abstract: To solve the difficulty in job scheduling in the complex and transient multi-user, multi-queue, and multi-data-center cloud computing environment, this paper proposed a job scheduling method based on deep reinforcement learning. A system model of cloud job scheduling and its mathematical model were built, and an optimization goal consisting of transmission time, waiting time, and execution time was obtained. A job scheduling algorithm based on deep reinforcement learning was designed, and its state space, action space, and reward function were given. A simulated cloud job scheduler was designed and developed, and simulated scheduling experiments were conducted on it. The …

A Data-Driven Modeling Method For Game Adversity Agent, Zeng Bi, Fang Xiao, Deshuai Kong, Xiangxiang Song, Zhengxuan Jia, Tingyu Lin

Journal of System Simulation

Abstract: Aiming at the problems of collaborative modeling of formation behavior and intelligent generation of decision-making in complex confrontation scenarios, based on the serious game to simulate the confrontation scenarios of complex maritime equipment against the air, this paper proposes a data-driven modeling method for game agent and uses a distributed modeling technology of parallel adversarial scenarios and opportunistic decision making technology of smart targets to achieve agent modeling. It provides support for the further exploration of multi-objective collaborative modeling in complex confrontation scenarios. The simulation results show that deep reinforcement learning algorithms can provide a basis for the modeling …

Research On Generation Technology Of Computer Generated Force In Lvc Training System, Gao Ang, Zhiming Dong, Guohui Zhang, Liang Tao, Qisheng Guo

Journal of System Simulation

Abstract: LVC training system of the combat equipment under the condition of confrontation is an effective means of training, aiming at the problem that in LVC training system, computer generated forces are difficult to meet the demand of training problems. The concept of LVC training and LVC training system is clarified, according to the relationship between model and system structure, the corresponding modeling technology requirements of three different hierarchical models, namely logical range entity configuration, command entity and combat entity, are expounded. According to the specific requirements, four computer-generated force generation methods are proposed, namely, logical target range virtual and …

Maidrl: Semi-Centralized Multi-Agent Reinforcement Learning Using Agent Influence, Anthony Lee Harris

Graduate Theses/Dissertations

In recent years, reinforcement learning algorithms, a subset of machine learning that focuses on solving problems through trial-and-error learning, have been used in the field of multi-agent systems to help the agents with interactions and cooperation on a variety of tasks. Given the enormous success of reinforcement learning in single-agent systems like Chess, Shogi, and Go, it is natural for the next step to be the expansion into multi-agent systems. However, controlling multiple agents simultaneously is extremely challenging, as the complexity increases tremendously with the number of agents in the system. Existing approaches in this regard use a wide range …

Empirical Studies Of Deep Learning On Information Diffusion On Social Networks And Collective Task Learning For Swarm Robotics, Trung T. Nguyen

Dissertations

Researchers in multiple disciplines have recently adopted deep learning because of its ability of high accuracy representation learning from big and complex data. My research goal in this thesis is developing deep learning models for information diffusion analysis on social networks and collective tasks learning in swarm robotics. Firstly, the information diffusion on social networks is modeled as a multivariate time series in three dimensions with ten features. Then, we applied time-series clustering algorithms with Dynamic Time Warping to discover different patterns of our models. Then, we build a prediction model based on LSTM, which outperforms traditional time-series prediction methods. …

Deep Reinforcement Learning For The Optimization Of Building Energy Control And Management, Jun Hao

Electronic Theses and Dissertations

Most of the current game-theoretic demand-side management methods focus primarily on the scheduling of home appliances, and the related numerical experiments are analyzed under various scenarios to achieve the corresponding Nash-equilibrium (NE) and optimal results. However, not much work is conducted for academic or commercial buildings. The methods for optimizing academic-buildings are distinct from the optimal methods for home appliances. In my study, we address a novel methodology to control the operation of heating, ventilation, and air conditioning system (HVAC).

We assume that each building in our campus is equipped with smart meter and communication system which is envisioned in …

Sentence Compression With Reinforcement Learning, Liangguo Wang, Jing Jiang, Lejian Liao

Research Collection School Of Computing and Information Systems

Deletion-based sentence compression is frequently formulated as a constrained optimization problem and solved by integer linear programming (ILP). However, ILP methods searching the best compression given the space of all possible compressions would be intractable when dealing with overly long sentences and too many constraints. Moreover, the hard constraints of ILP would restrict the available solutions. This problem could be even more severe considering parsing errors. As an alternative solution, we formulate this task in a reinforcement learning framework, where hard constraints are used as rewards in a soft manner. The experiment results show that our method achieves competitive performance …