Operations Research, Systems Engineering and Industrial Engineering Commons^™

Policy Gradient With Value Function Approximation For Collective Multiagent Planning, Duc Thien Nguyen, Akshat Kumar, Hoong Chuin Lau

Research Collection School Of Computing and Information Systems

Decentralized (PO)MDPs provide an expressive framework for sequential decision making in a multiagent system. Given their computational complexity, recent research has focused on tractable yet practical subclasses of Dec-POMDPs. We address such a subclass called CDec-POMDP where the collective behavior of a population of agents affects the joint-reward and environment dynamics. Our main contribution is an actor-critic (AC) reinforcement learning method for optimizing CDec-POMDP policies. Vanilla AC has slow convergence for larger problems. To address this, we show how a particular decomposition of the approximate action-value function over agents leads to effective updates, and also derive a new way to …

A Selective-Discrete Particle Swarm Optimization Algorithm For Solving A Class Of Orienteering Problems, Aldy Gunawan, Vincent F. Yu, Perwira Redi, Parida Jewpanya, Hoong Chuin Lau

Research Collection School Of Computing and Information Systems

This study addresses a class of NP-hard problem called the Orienteering Problem (OP), which belongs to a well-known class of vehicle routing problems. In the OP, a set of nodes that associated with a location and a score is given. The time required to travel between each pair of nodes is known in advance. The total travel time is limited by a predetermined time budget. The objective is to select a subset of nodes to be visited that maximizes the total collected score within a path. The Team OP (TOP) is an extension of OP that incorporates multiple paths. Another …

Efficient Gate System Operations For A Multipurpose Port Using Simulation Optimization, Ketki Kulkarni, Trong Khiem Tran, Hai Wang, Hoong Chuin Lau

Research Collection School Of Computing and Information Systems

Port capacity is determined by three major infrastructural resources namely, berths, yards and gates. Theadvertised capacity is constrained by the least of the capacities of the three resources. While a lot ofattention has been paid to optimizing berth and yard capacities, not much attention has been given toanalyzing the gate capacity. The gates are a key node between the land-side and sea-side operations in anocean-to-cities value chain. The gate system under consideration, located at an important port in an Asiancity, is a multi-class parallel queuing system with non-homogeneous Poisson arrivals. It is hard to obtaina closed form analytic approach for …

A Unified Framework For Vehicle Rerouting And Traffic Light Control To Reduce Traffic Congestion, Zhiguang Cao, Siwei Jiang, Jie Zhang, Hongliang Guo

Research Collection School Of Computing and Information Systems

As the number of vehicles grows rapidly each year, more and more traffic congestion occurs, becoming a big issue for civil engineers in almost all metropolitan cities. In this paper, we propose a novel pheromone-based traffic management framework for reducing traffic congestion, which unifies the strategies of both dynamic vehicle rerouting and traffic light control. Specifically, each vehicle, represented as an agent, deposits digital pheromones over its route, while roadside infrastructure agents collect the pheromones and fuse them to evaluate real-time traffic conditions as well as to predict expected road congestion levels in near future. Once road congestion is predicted, …

A Multi-Agent System For Coordinating Vessel Traffic, Teck-Hou Teng, Hoong Chuin Lau, Akshat Kumar

Research Collection School Of Computing and Information Systems

Environmental, regulatory and resource constraints affects the safety and efficiency of vessels navigating in and out of the ports. Movement of vessels under such constraints must be coordinated for improving safety and efficiency. Thus, we frame the vessel coordination problem as a multi-agent path-finding (MAPF) problem. We solve this MAPF problem using a Coordinated Path-Finding (CPF) algorithm. Based on the local search paradigm, the CPF algorithm improves on the aggregated path quality of the vessels iteratively. Outputs of the CPF algorithm are the coordinated trajectories. The Vessel Coordination Module (VCM) described here is the module encapsulating our MAPF-based approach for …

Decentralized Planning In Stochastic Environments With Submodular Rewards, Rajiv Ranjan Kumar, Pradeep Varakantham, Akshat Kumar

Research Collection School Of Computing and Information Systems

Decentralized Markov Decision Process (Dec-MDP) providesa rich framework to represent cooperative decentralizedand stochastic planning problems under transition uncertainty.However, solving a Dec-MDP to generate coordinatedyet decentralized policies is NEXP-Hard. Researchershave made significant progress in providing approximate approachesto improve scalability with respect to number ofagents. However, there has been little or no research devotedto finding guarantees on solution quality for approximateapproaches considering multiple (more than 2 agents)agents. We have a similar situation with respect to the competitivedecentralized planning problem and the StochasticGame (SG) model. To address this, we identify models in thecooperative and competitive case that rely on submodular rewards,where we show …

An Efficient Approach To Model-Based Hierarchical Reinforcement Learning, Zhuoru Li, Akshay Narayan, Tze-Yun Leong

Research Collection School Of Computing and Information Systems

We propose a model-based approach to hierarchical reinforcement learning that exploits shared knowledge and selective execution at different levels of abstraction, to efficiently solve large, complex problems. Our framework adopts a new transition dynamics learning algorithm that identifies the common action-feature combinations of the subtasks, and evaluates the subtask execution choices through simulation. The framework is sample efficient, and tolerates uncertain and incomplete problem characterization of the subtasks. We test the framework on common benchmark problems and complex simulated robotic environments. It compares favorably against the stateof-the-art algorithms, and scales well in very large problems.

Dynamic Repositioning To Reduce Lost Demand In Bike Sharing Systems, Supriyo Ghosh, Pradeep Varakantham, Yossiri Adulyasak, Patrick Jaillet

Research Collection School Of Computing and Information Systems

Bike Sharing Systems (BSSs) are widely adopted in major cities of the world due to concerns associated with extensive private vehicle usage, namely, increased carbon emissions, traffic congestion and usage of nonrenewable resources. In a BSS, base stations are strategically placed throughout a city and each station is stocked with a pre-determined number of bikes at the beginning of the day. Customers hire the bikes from one station and return them at another station. Due to unpredictable movements of customers hiring bikes, there is either congestion (more than required) or starvation (fewer than required) of bikes at base stations. Existing …

Sensitivity Analysis Method To Address User Disparities In The Analytic Hierarchy Process, Marie Ivanco, Gene Hou, Jennifer Michaeli

Mechanical & Aerospace Engineering Faculty Publications

Decision makers often face complex problems, which can seldom be addressed well without the use of structured analytical models. Mathematical models have been developed to streamline and facilitate decision making activities, and among these, the Analytic Hierarchy Process (AHP) constitutes one of the most utilized multi-criteria decision analysis methods. While AHP has been thoroughly researched and applied, the method still shows limitations in terms of addressing user profile disparities. A novel sensitivity analysis method based on local partial derivatives is presented here to address these limitations. This new methodology informs AHP users of which pairwise comparisons most impact the derived …

A New Reinforcement Learning Algorithm With Fixed Exploration For Semi-Markov Decision Processes, Angelo Michael Encapera

Masters Theses

"Artificial intelligence or machine learning techniques are currently being widely applied for solving problems within the field of data analytics. This work presents and demonstrates the use of a new machine learning algorithm for solving semi-Markov decision processes (SMDPs). SMDPs are encountered in the domain of Reinforcement Learning to solve control problems in discrete-event systems. The new algorithm developed here is called iSMART, an acronym for imaging Semi-Markov Average Reward Technique. The algorithm uses a constant exploration rate, unlike its precursor R-SMART, which required exploration decay. The major difference between R-SMART and iSMART is that the latter uses, in addition …

A Bounded Actor-Critic Algorithm For Reinforcement Learning, Ryan Jacob Lawhead

Masters Theses

"This thesis presents a new actor-critic algorithm from the domain of reinforcement learning to solve Markov and semi-Markov decision processes (or problems) in the field of airline revenue management (ARM). The ARM problem is one of control optimization in which a decision-maker must accept or reject a customer based on a requested fare. This thesis focuses on the so-called single-leg version of the ARM problem, which can be cast as a semi-Markov decision process (SMDP). Large-scale Markov decision processes (MDPs) and SMDPs suffer from the curses of dimensionality and modeling, making it difficult to create the transition probability matrices (TPMs) …

Human-Intelligence/Machine-Intelligence Decision Governance: An Analysis From Ontological Point Of View, Faisal Mahmud, Teddy Steven Cotter

Engineering Management & Systems Engineering Faculty Publications

The increasing CPU power and memory capacity of computers, and now computing appliances, in the 21st century has allowed accelerated integration of artificial intelligence (AI) into organizational processes and everyday life. Artificial intelligence can now be found in a wide range of organizational processes including medical diagnosis, automated stock trading, integrated robotic production systems, telecommunications routing systems, and automobile fuzzy logic controllers. Self-driving automobiles are just the latest extension of AI. This thrust of AI into organizations and everyday life rests on the AI community’s unstated assumption that “…every aspect of human learning and intelligence could be so precisely described …

Cognition-Based Approaches For High-Precision Text Mining, George John Shannon

Doctoral Dissertations

"This research improves the precision of information extraction from free-form text via the use of cognitive-based approaches to natural language processing (NLP). Cognitive-based approaches are an important, and relatively new, area of research in NLP and search, as well as linguistics. Cognitive approaches enable significant improvements in both the breadth and depth of knowledge extracted from text. This research has made contributions in the areas of a cognitive approach to automated concept recognition in.

Cognitive approaches to search, also called concept-based search, have been shown to improve search precision. Given the tremendous amount of electronic text generated in our digital …

Managing Egress Of Crowd During Infrastructure Disruption, Teck Hou Teng, Shih-Fen Cheng, Trong-Nghia Truong, Hoong Chuin Lau

Research Collection School Of Computing and Information Systems

In a large indoor environment such as a sports arena or convention center, smooth egress of crowd after an event can be seriously affected if infrastructure such as elevators and escalators break down. In this paper, we propose a novel crowd simulator known as SIM-DISRUPT for simulating egress scenarios in non-emergency situations. To surface the impact of disrupted infrastructure on the egress of crowd, SIM-DISRUPT includes features that allow users to specify selective disruptions as well as strategies for controlling the distribution and egress choices of crowd. Using SIM-DISRUPT, we investigate effects of crowd distribution, egress choices and infrastructure disruptions …

Orienteering Problem: A Survey Of Recent Variants, Solution Approaches And Applications, Aldy Gunawan, Hoong Chuin Lau, Pieter Vansteenwegen

Research Collection School Of Computing and Information Systems

Duplicate record, see https://ink.library.smu.edu.sg/sis_research/3271. The Orienteering Problem (OP) has received a lot of attention in the past few decades. The OP is a routing problem in which the goal is to determine a subset of nodes to visit, and in which order, so that the total collected score is maximized and a given time budget is not exceeded. A number of typical variants has been studied, such as the Team OP, the (Team) OP with Time Windows and the Time Dependent OP. Recently, a number of new variants of the OP was introduced, such as the Stochastic OP, the …

Orienteering Problem: A Survey Of Recent Variants, Solution Approaches And Applications, Aldy Gunawan, Hoong Chuin Lau, Pieter Vansteenwegen

Research Collection School Of Computing and Information Systems

The Orienteering Problem (OP) has received a lot of attention in the past few decades. The OP is a routing problem in which the goal is to determine a subset of nodes to visit, and in which order, so that the total collected score is maximized and a given time budget is not exceeded. A number of typical variants has been studied, such as the Team OP, the (Team) OP with Time Windows and the Time Dependent OP. Recently, a number of new variants of the OP was introduced, such as the Stochastic OP, the Generalized OP, the Arc OP, …

An Agent-Based Approach To Human Migration Movement, Larry Lin, Kathleen M. Carley, Shih-Fen Cheng

Research Collection School Of Computing and Information Systems

How are the populations of the world likely to shift? Which countries will be impacted by sea-level rise? This paper uses a country-level agent-based dynamic network model to examine shifts in population given network relations among countries, which influences overall population change. Some of the networks considered include: alliance networks, shared language networks, economic influence networks, and proximity networks. Validation of model is done for migration probabilities between countries, as well as for country populations and distributions. The proposed framework provides a way to explore the interaction between climate change and policy factors at a global scale.

Enhancing Local Search With Adaptive Operator Ordering And Its Application To The Time Dependent Orienteering Problem, Aldy Gunawan, Hoong Chuin Lau, Kun Lu

Research Collection School Of Computing and Information Systems

No abstract provided.

A Hybrid Tabu/Scatter Search Algorithm For Simulation-Based Optimization Of Multi-Objective Runway Operations Scheduling, Bulent Soykan

Engineering Management & Systems Engineering Theses & Dissertations

As air traffic continues to increase, air traffic flow management is becoming more challenging to effectively and efficiently utilize airport capacity without compromising safety, environmental and economic requirements. Since runways are often the primary limiting factor in airport capacity, runway operations scheduling emerge as an important problem to be solved to alleviate flight delays and air traffic congestion while reducing unnecessary fuel consumption and negative environmental impacts. However, even a moderately sized real-life runway operations scheduling problem tends to be too complex to be solved by analytical methods, where all mathematical models for this problem belong to the complexity class …

Dual Formulations For Optimizing Dec-Pomdp Controllers, Akshat Kumar, Hala Mostafa, Shlomo Zilberstein

Research Collection School Of Computing and Information Systems

Decentralized POMDP is an expressive model for multi-agent planning. Finite-state controllers (FSCs)---often used to represent policies for infinite-horizon problems---offer a compact, simple-to-execute policy representation. We exploit novel connections between optimizing decentralized FSCs and the dual linear program for MDPs. Consequently, we describe a dual mixed integer linear program (MIP) for optimizing deterministic FSCs. We exploit the Dec-POMDP structure to devise a compact MIP and formulate constraints that result in policies executable in partially-observable decentralized settings. We show analytically that the dual formulation can also be exploited within the expectation maximization (EM) framework to optimize stochastic FSCs. The resulting EM algorithm …

Self-Organizing Neural Network For Adaptive Operator Selection In Evolutionary Search, Teck Hou Teng, Stephanus Daniel Handoko, Hoong Chuin Lau

Research Collection School Of Computing and Information Systems

Evolutionary Algorithm is a well-known meta-heuristics paradigm capable of providing high-quality solutions to computationally hard problems. As with the other meta-heuristics, its performance is often attributed to appropriate design choices such as the choice of crossover operators and some other parameters. In this chapter, we propose a continuous state Markov Decision Process model to select crossover operators based on the states during evolutionary search. We propose to find the operator selection policy efficiently using a self-organizing neural network, which is trained offline using randomly selected training samples. The trained neural network is then verified on test instances not used for …

Strategic Planning For Setting Up Base Stations In Emergency Medical Systems, Supriyo Ghosh, Pradeep Varakantham

Research Collection School Of Computing and Information Systems

Emergency Medical Systems (EMSs) are an important component of public health-care services. Improving infrastructure for EMS and specifically the construction of base stations at the ”right” locations to reduce response times is the main focus of this paper. This is a computationally challenging task because of the: (a) exponentially large action space arising from having to consider combinations of potential base locations, which themselves can be significant; and (b) direct impact on the performance of the ambulance allocation problem, where we decide allocation of ambulances to bases. We present an incremental greedy approach to discover the placement of bases that …

Designing And Comparing Multiple Portfolios Of Parameter Configurations For Online Algorithm Selection, Aldy Gunawan, Hoong Chuin Lau, Mustafa Misir

Research Collection School Of Computing and Information Systems

Algorithm portfolios seek to determine an effective set of algorithms that can be used within an algorithm selection framework to solve problems. A limited number of these portfolio studies focus on generating different versions of a target algorithm using different parameter configurations. In this paper, we employ a Design of Experiments (DOE) approach to determine a promising range of values for each parameter of an algorithm. These ranges are further processed to determine a portfolio of parameter configurations, which would be used within two online Algorithm Selection approaches for solving different instances of a given combinatorial optimization problem effectively. We …

Simultaneous Optimization And Sampling Of Agent Trajectories Over A Network, Hala Mostafa, Akshat Kumar, Hoong Chuin Lau

Research Collection School Of Computing and Information Systems

We study the problem of optimizing the trajectories of agents moving over a network given their preferences over which nodes to visit subject to operational constraints on the network. In our running example, a theme park manager optimizes which attractions to include in a day-pass to maximize the pass’s appeal to visitors while keeping operational costs within budget. The first challenge in this combinatorial optimization problem is that it involves quantities (expected visit frequencies of each attraction) that cannot be expressed analytically, for which we use the Sample Average Approximation. The second challenge is that while sampling is typically done …

Approximate Inference Using Dc Programming For Collective Graphical Models, Duc Thien Nguyen, Akshat Kumar, Hoong Chuin Lau, Daniel Sheldon

Research Collection School Of Computing and Information Systems

Collective graphical models (CGMs) provide a framework for reasoning about a population of independent and identically distributed individuals when only noisy and aggregate observations are given. Previous approaches for inference in CGMs work on a junction-tree representation, thereby highly limiting their scalability. To remedy this, we show how the Bethe entropy approximation naturally arises for the inference problem in CGMs. We reformulate the resulting optimization problem as a difference-of-convex functions program that can capture different types of CGM noise models. Using the concave-convex procedure, we then develop a scalable message-passing algorithm. Empirically, our approach is highly scalable and accurate for …

Reinforcement Learning Framework For Modeling Spatial Sequential Decisions Under Uncertainty: (Extended Abstract), Truc Viet Le, Siyuan Liu, Hoong Chuin Lau

Research Collection School Of Computing and Information Systems

We consider the problem of trajectory prediction, where a trajectory is an ordered sequence of location visits and corresponding timestamps. The problem arises when an agent makes sequential decisions to visit a set of spatial locations of interest. Each location bears a stochastic utility and the agent has a limited budget to spend. Given the agent's observed partial trajectory, our goal is to predict the remaining trajectory. We propose a solution framework to the problem considering both the uncertainty of utility and the budget constraint. We use reinforcement learning (RL) to model the underlying decision processes and inverse RL to …

Robust Influence Maximization, Meghna Lowalekar, Pradeep Varakantham, Akshat Kumar

Research Collection School Of Computing and Information Systems

Influence Maximization is the problem of finding a fixed size set of nodes, which will maximize the expected number of influenced nodes in a social network. The number of influenced nodes is dependent on the influence strength of edges that can be very noisy. The noise in the influence strengths can be modeled using a random noise or adversarial noise model. It has been shown that all random processes that independently affect edges of the graph can be absorbed into the activation probabilities themselves and hence random noise can be captured within the independent cascade model. On the other hand, …

Shortest Path Based Decision Making Using Probabilistic Inference, Akshat Kumar

Research Collection School Of Computing and Information Systems

We present a new perspective on the classical shortest path routing (SPR) problem in graphs. We show that the SPR problem can be recast to that of probabilistic inference in a mixture of simple Bayesian networks. Maximizing the likelihood in this mixture becomes equivalent to solving the SPR problem. We develop the well known Expectation-Maximization (EM) algorithm for the SPR problem that maximizes the likelihood, and show that it does not get stuck in a locally optimal solution. Using the same probabilistic framework, we then address an NP-Hard network design problem where the goal is to repair a network of …

The Reconfigurable Machinery Efficient Workspace Analysis Based On The Twist Angles, Ana M. Djuric, Vukica Jovanovic, Mirjana Filipovic, Ljubinko Kevac

Engineering Technology Faculty Publications

A novel methodology for the calculation, visualisation and analysis of the Reconfigurable Machinery Efficient Workspace (RMEW), based on the twist angles, is presented in this paper. The machinery's kinematic parameters are used for calculating the workspace, while the efficient workspace is associated with the machinery's path and includes the end-effector position and orientation. To analyse and visualise many different machinery efficient workspaces at the same time, the calculation is based on the previously developed and validated complex reconfigurable machinery's kinematic structure named n-DOF Global Kinematic Model (n-GKM). An industrial robot is used as an example to demonstrate …

Robust Execution Strategies For Project Scheduling With Unreliable Resources And Stochastic Durations, Na Fu, Hoong Chuin Lau, Pradeep Varakantham

Research Collection School Of Computing and Information Systems

The resource-constrained project scheduling problem with minimum and maximum time lags (RCPSP/max) is a general model for resource scheduling in many real-world problems (such as manufacturing and construction engineering). We consider RCPSP/max problems where the durations of activities are stochastic and resources can have unforeseen breakdowns. Given a level of allowable risk, (Formula presented.), our mechanisms aim to compute the minimum robust makespan execution strategy. Robust makespan for an execution strategy is any makespan value that has a risk less than (Formula presented.). The risk for a makespan value, (Formula presented.) given an execution strategy, is the probability that a …