Open Access. Powered by Scholars. Published by Universities.®
Articles 1 - 9 of 9
Full-Text Articles in Entire DC Network
An Introspective Approach For Competence-Aware Autonomy, Connor Basich
An Introspective Approach For Competence-Aware Autonomy, Connor Basich
Doctoral Dissertations
Building and deploying autonomous systems in the open world has long been a goal of both the artificial intelligence (AI) and robotics communities. From autonomous driving, to health care, to office assistance, these systems have the potential to transform society and alter our everyday lives. The open world, however, presents numerous challenges that question the typical assumptions made by the models and frameworks often used in contemporary AI and robotics. Systems in the open world are faced with an unconstrained and non-stationary environment with a range of heterogeneous actors that is too complex to be modeled in its entirety. Moreover, …
Scheduling Allocation And Inventory Replenishment Problems Under Uncertainty: Applications In Managing Electric Vehicle And Drone Battery Swap Stations, Amin Asadi
Graduate Theses and Dissertations
In this dissertation, motivated by electric vehicle (EV) and drone application growth, we propose novel optimization problems and solution techniques for managing the operations at EV and drone battery swap stations. In Chapter 2, we introduce a novel class of stochastic scheduling allocation and inventory replenishment problems (SAIRP), which determines the recharging, discharging, and replacement decisions at a swap station over time to maximize the expected total profit. We use Markov Decision Process (MDP) to model SAIRPs facing uncertain demands, varying costs, and battery degradation. Considering battery degradation is crucial as it relaxes the assumption that charging/discharging batteries do not …
Solving Online Threat Screening Games Using Constrained Action Space Reinforcement Learning, Sanket Shah, Arunesh Sinha, Pradeep Varakantham, Andrew Perrault, Millind Tambe
Solving Online Threat Screening Games Using Constrained Action Space Reinforcement Learning, Sanket Shah, Arunesh Sinha, Pradeep Varakantham, Andrew Perrault, Millind Tambe
Research Collection School Of Computing and Information Systems
Large-scale screening for potential threats with limited resources and capacity for screening is a problem of interest at airports, seaports, and other ports of entry. Adversaries can observe screening procedures and arrive at a time when there will be gaps in screening due to limited resource capacities. To capture this game between ports and adversaries, this problem has been previously represented as a Stackelberg game, referred to as a Threat Screening Game (TSG). Given the significant complexity associated with solving TSGs and uncertainty in arrivals of customers, existing work has assumed that screenees arrive and are allocated security resources at …
Landing Throttleable Hybrid Rockets With Hierarchical Reinforcement Learning In A Simulated Environment, Francesco Alessandro Stefano Mikulis-Borsoi
Landing Throttleable Hybrid Rockets With Hierarchical Reinforcement Learning In A Simulated Environment, Francesco Alessandro Stefano Mikulis-Borsoi
Honors Theses and Capstones
In this paper, I develop a hierarchical Markov Decision Process (MDP) structure for completing the task of vertical rocket landing. I start by covering the background of this problem, and formally defining its constraints. In order to reduce mistakes while formulating different MDPs, I define and develop the criteria for a standardized MDP definition format. I then decompose the problem into several sub-problems of vertical landing, namely velocity control and vertical stability control. By exploiting MDP coupling and symmetrical properties, I am able to significantly reduce the size of the state space compared to a unified MDP formulation. This paper …
Pond-Hindsight: Applying Hindsight Optimization To Partially-Observable Markov Decision Processes, Alan Olsen
Pond-Hindsight: Applying Hindsight Optimization To Partially-Observable Markov Decision Processes, Alan Olsen
All Graduate Theses and Dissertations, Spring 1920 to Summer 2023
Partially-observable Markov decision processes (POMDPs) are especially good at modeling real-world problems because they allow for sensor and effector uncertainty. Unfortunately, such uncertainty makes solving a POMDP computationally challenging. Traditional approaches, which are based on value iteration, can be slow because they find optimal actions for every possible situation. With the help of the Fast Forward (FF) planner, FF- Replan and FF-Hindsight have shown success in quickly solving fully-observable Markov decision processes (MDPs) by solving classical planning translations of the problem. This thesis extends the concept of problem determination to POMDPs by sampling action observations (similar to how FF-Replan samples …
Optimal Adaptation In Web Processes With Coordination Constraints, Kunal Verma, Prashant Doshi, Karthik Gomadam, John A. Miller, Amit P. Sheth
Optimal Adaptation In Web Processes With Coordination Constraints, Kunal Verma, Prashant Doshi, Karthik Gomadam, John A. Miller, Amit P. Sheth
Kno.e.sis Publications
We present methods for optimally adapting Web processes to exogenous events while preserving inter-service constraints that necessitate coordination. For example, in a supply chain process, orders placed by a manufacturer may get delayed in arriving. In response to this event, the manufacturer has the choice of either waiting out the delay or changing the supplier. Additionally, there may be compatibility constraints between the different orders, thereby introducing the problem of coordination between them if the manufacturer chooses to change the suppliers. We focus on formulating the decision making models of the managers, who must adapt to external events while satisfying …
Optimal Adaptation In Autonomic Web Processes With Inter-Service Dependencies, Kunal Verma, Prashant Doshi, Karthik Gomadam, John A. Miller, Amit P. Sheth
Optimal Adaptation In Autonomic Web Processes With Inter-Service Dependencies, Kunal Verma, Prashant Doshi, Karthik Gomadam, John A. Miller, Amit P. Sheth
Kno.e.sis Publications
We present methods for optimally adapting Web processes to exogenous events while preserving inter-service dependencies. For example, in a supply chain process, orders placed by the manufacturer may get delayed in arriving. In response to this event, the manufacturer has the choice of either waiting out the delay or changing the supplier.
Prioritization Methods For Accelerating Mdp Solvers, Kevin Seppi, David Wingate
Prioritization Methods For Accelerating Mdp Solvers, Kevin Seppi, David Wingate
Faculty Publications
The performance of value and policy iteration can be dramatically improved by eliminating redundant or useless backups, and by backing up states in the right order. We study several methods designed to accelerate these iterative solvers, including prioritization, partitioning, and variable reordering. We generate a family of algorithms by combining several of the methods discussed, and present extensive empirical evidence demonstrating that performance can improve by several orders of magnitude for many problems, while preserving accuracy and convergence guarantees.
Solving Large Mdps Quickly With Partitioned Value Iteration, David Wingate
Solving Large Mdps Quickly With Partitioned Value Iteration, David Wingate
Theses and Dissertations
Value iteration is not typically considered a viable algorithm for solving large-scale MDPs because it converges too slowly. However, its performance can be dramatically improved by eliminating redundant or useless backups, and by backing up states in the right order. We present several methods designed to help structure value dependency, and present a systematic study of companion prioritization techniques which focus computation in useful regions of the state space. In order to scale to solve ever larger problems, we evaluate all enhancements and methods in the context of parallelizability. Using the enhancements, we discover that in many instances the limiting …