Control Theory Commons^™

Finite Hybrid- And Semi-Markov Chains, Jose L. Menaldi, Maurice Robin

Mathematics Faculty Research Publications

The ergodic behaviour of finite Markov chains having also instantaneous transition are considered. Some estimates are obtained which complement our previous work [29]. Also, an optimal switching control model for semi-Markov chains is analysed, without any particular assumptions on the recurrent classes.

Ergodic Switching Control For Markov-Feller Processes Ii, Jose L. Menaldi, Maurice Robin

Mathematics Faculty Research Publications

This is the continuation of Part I [14], where we considered control problems with long term average (or ergodic) cost for Markov switching processes (zt , nt ), nt being a discrete component with values in a finite set N . The control acts only on this discrete component and consists of immediate switching actions. We solve the ergodic problem in several situations extending previous works, mainly when zt is a reflected diffusion with or without jumps and when the set of control values is strictly smaller than N . In this Part II, we conclude our theoretical analysis with …

Ergodic Switching Control For Markov-Feller Processes I, Jose L. Menaldi, Maurice Robin

Mathematics Faculty Research Publications

We consider control problems with long term average (or ergodic) cost for Markov switching processes (zt, nt ), nt being a discrete component with values in a finite set N . The control acts only on this discrete component and consists of immediate switching actions. We solve the ergodic problem in several situations extending previous works, mainly when zt is a reflected diffusion with or without jumps and when the set of control values is strictly smaller than N .

Optimal Stopping Problems For A Family Of Continuous-Time Markov Processes, Héctor Jasso-Fuentes, Jose-Luis Menaldi, Fidel Vásquez-Rojas

Mathematics Faculty Research Publications

In this paper we study the well-know optimal stopping problem applied to a general family of continuous-time Markov process. The approach to follow is merely analytic and it is based on the characterization of stopping problems through the study of a certain variational inequality; namely one solution of this inequality will coincide with the optimal value of the stopping problem. In addition, by means of this characterization, it is possible to find the so-named continuation region, and as a byproduct obtaining the optimal stopping time. The most of the material is based on the semigroup theory, infinitesimal generators and resolvents. …

Basic Probability Theory, Jose Luis Menaldi

Mathematics Faculty Research Publications

Long title: Basic Probability Theory: Independent Random Variables and Sample Spaces. Chapters: Elementary Probability - Basic Probability - Canonical Sample Spaces - Working on Probability Spaces - A Solutions to Exercises.

Relaxation And Linear Programs On A Hybrid Control Model, Héctor Jasso-Fuentes, Jose-Luis Menaldi

Mathematics Faculty Research Publications

Some optimality results for hybrid control problems are presented. The hybrid model under study consists of two subdynamics, one of a standard type governed by an ordinary differential equation, and the other of a special type having a discrete evolution. We focus on the case when the interaction between the subdynamics takes place only when the state of the system reaches a given fixed region of the state space. The controller is able to apply two controls, each applied to one of the two subdynamics, whereas the state follows a composite evolution, of continuous type and discrete type. By the …

On Optimal Stopping And Impulse Control With Constraint, J. L. Menaldi, M. Robin

Mathematics Faculty Research Publications

The optimal stopping and impulse control problems for a Markov-Feller process are considered when the controls are allowed only when a signal arrives. This is referred to as control problems with constraint. In [28, 29, 30], the HJB equation was solved and an optimal control (for the optimal stopping problem, the discounted impulse control problem and the ergodic impulse control problem, respectively) was obtained, under suitable conditions, including a setting on a compact metric state space. In this work, we extend most of the results to the situation where the state space of the Markov process is locally compact.

On Some Ergodic Impulse Control Problems With Constraint, J. L. Menaldi, Maurice Robin

Mathematics Faculty Research Publications

This paper studies the impulse control of a general Markov process under the average (or ergodic) cost when the impulse instants are restricted to be the arrival times of an exogenous process, and this restriction is referred to as a constraint. A detailed setting is described, a characterization of the optimal cost is obtained as a solution of an HJB equation, and an optimal impulse control is identified.

Discrete-Time Hybrid Control In Borel Spaces: Average Cost Optimality Criterion, Héctor Jasso-Fuentes, José-Luis Menaldi, Tomás Prieto-Rumeau, Maurice Robin

Mathematics Faculty Research Publications

This paper addresses an optimal hybrid control problem in discrete-time with Borel state and action spaces. By hybrid we mean that the evolution of the state of the system may undergo deep changes according to structural modifications of the dynamic. Such modifications occur either by the position of the state or by means of the controller's actions. The optimality criterion is of a long-run ratio-average (or ratio-ergodic) type. We provide the existence of optimal average policies for this hybrid control problem by analyzing an associated dynamic programming equation. We also show that this problem can be translated into a standard …

Discrete-Time Hybrid Control In Borel Spaces, Héctor Jasso-Fuentes, José-Luis Menaldi, Tomás Prieto-Rumeau

Mathematics Faculty Research Publications

A discrete-time hybrid control model with Borel state and action spaces is introduced. In this type of models, the dynamic of the system is composed by two sub-dynamics affecting the evolution of the state; one is of a standard-type that runs almost every time and another is of a special-type that is active under special circumstances. The controller is able to use two different type of actions, each of them is applied to each of the two sub-dynamics, and the activations of these sub-dynamics are possible according to an activation rule that can be handled by the controller. The aim …

Sdes, Jumps And Estimates, Jose L. Menaldi

Mathematics Faculty Research Publications

Long Title: Stochastic Ordinary Differential Equations with Jumps: Theory and Estimates. Chapters: Stochastic Integrals - Initial Approach to SDEs - Estimates of SDEs - Other Formulations of SDEs - SDEs with Reflection - PDE Connections.

On Some Impulse Control Problems With Constraint, Jose L. Menaldi, Maurice Robin

Mathematics Faculty Research Publications

The impulse control of a Markov–Feller process is considered when the impulses are allowed only when a signal arrives. This is referred to as an impulse control problem with constraint. A detailed setting is described, a characterization of the optimal cost is obtained using previous results of the authors on optimal stopping problems with constraint, and an optimal impulse control is identified.

Optimal Control And Feedback Design Of State-Constrained Parabolic Systems In Uncertainty Conditions, Boris S. Mordukhovich

Mathematics Research Reports

The paper concerns minimax control problems for linear multidimensional parabolic systems with distributed uncertain perturbations and control functions acting in the Dirichlet boundary conditions. The underlying parabolic control system is functioning under hard/pointwise constraints on control and state variables. The main goal is to design a feedback control regulator that ensures the required state performance and robust stability under any feasible perturbations and minimize an energy-type functional under the worst perturbations from the given area. We develop a constructive approach to the minimax control design of constrained parabolic systems that is based on certain characteristic features of the parabolic dynamics …

Stochastic Differential Equations With Jumps, Jose L. Menaldi

Mathematics Faculty Research Publications

Part I Stochastic Processes with Jumps Chapters: Probability Spaces, Semigroup Theory - Part II Stochastic Differential Equations with Jumps Chapters: Stochastic Calculus, Stochastic Differential Equations - Part III Reflected SDE with Jumps Chapters: Stochastic Differential Equations II, Stochastic Differential Equations III.

Comment: This is last version from 2014-01-07. *This Initial version 15/May/2008 was corrected and augmented to produce the others 5 volumes.

Optimization And Feedback Design Of State-Constrained Parabolic Systems, Boris S. Mordukhovich

Mathematics Research Reports

The paper is devoted to optimal control and feedback design of stateconstrained parabolic systems in uncertainty conditions. Problems of this type are among the most challenging and difficult in dynamic optimization for any kind of dynamical systems. We pay the main attention to considering linear multidimensional parabolic'systems with Dirichlet boundary controls and pointwise state constraints, while the methods developed in this study are applicable to other kinds of boundary controls and dynamical systems of the parabolic type. The feedback design problem is formulated in the minimax sense to ensure stabilization of transients within the prescribed diapason and robust stability of …

Suboptimal Feedback Control Design Of Constrained Parabolic Systems In Uncertainty Conditions, Boris S. Mordukhovich

Mathematics Research Reports

The paper concerns minimax control problems forlinear multidimensional parabolic systems with distributed uncertain perturbations and control functions acting in the Dirichlet boundary conditions. The underlying parabolic control system is functioning under hard/pointwise constraints on control and state variables. The main goal is to design a feedback control regulator that ensures the required state performance and robust stability under any feasible perturbations and minimize an energy-type functional under the worst perturbations from the given area. We develop an efficient approach to the minimax control design of constrained parabolic systems that is based on certain characteristic features of the parabolic dynamics including …

Optimal Control Of Delay Systems With Differential And Algebraic Dynamic Constraints, Boris S. Mordukhovich, Lianwen Wang

Mathematics Research Reports

This paper concerns constrained dynamic optimization problems governed by delay control systems whose dynamic constraints are described by both delay-differential inclusions and linear algebraic equations. This is a new class of optimal control systems that, on one hand, may be treated as a specific type of variational problems for neutral functional-differential inclusions while, on the other hand, is related to a special class of differential-algebraic systems with a general delay-differential inclusion and a linear constraint link between "slow" and "fast" variables. We pursue a two-hold goal: to study variational stability for this class of control systems with respect to discrete …

Optimal Control Of Delayed Differential-Algebraic Inclusions, Boris S. Mordukhovich, Lianwen Wang

Mathematics Research Reports

This paper concerns constrained dynamic optimization problems governed by delayed differential-algebraic systems. Dynamic constraints in such systems, which are particularly important for engineering applications, are described by interconnected delay-differential inclusions and algebraic equations. We pursue a two-hold goal: to study variational stability of such control systems with respect to discrete approximations and to derive necessary optimality conditions for both delayed differential-algebraic systems and their finite-difference counterparts using modern tools of variational analysis and generalized differentiation. We are not familiar with any results in these directions for differential-algebraic inclusions even in the delay-free case. In the first part of the paper …

The Approximate Maxium Principle In Constrained Optimal Control, Boris S. Mordukhovich, Ilya Shvartsman

Mathematics Research Reports

The paper concerns optimal control problems for dynamic systems governed by a parametric family of discrete approximations of control systems with continuous time. Discrete approximations play an important role in both qualitative and numerical aspects of optimal control and occupy an intermediate position between discrete-time and continuous-time control systems. The central result in optimal control of discrete approximations is the Approximate Maximum Principle (AMP), which is justified for smooth control problems with endpoint constraints under certain assumptions without imposing any convexity, in contrast to discrete systems with a fixed step. We show that these assumptions are essential for the validity …

Neumann Boundary Control Of Hyperbolic Equations With Pointwise State Constraints, Boris S. Mordukhovich, Jean-Pierre Raymond

Mathematics Research Reports

We consider optimal control problems for hyperbolic systems with controls in Neumann boundary conditions with pointwise (hard) constraints on control and state functions. Focusing on hyperbolic dynamics governed by the multidimensional wave equation with a nonlinear term, we derive new necessary optimality conditions in the pointwise form of the Pontryagin Maximum Principle for the state-constrained problem under consideration. Our approach is based on modern methods of variational analysis that allows us to obtain refined necessary optimality conditions with no convexity assumptions on integrands in the minimizing cost functional.

Dirichlet Boundary Control Of Hyperbolic Equations In The Presence Of State Constraints, Boris S. Mordukhovich, Jean-Pierre Raymond

Mathematics Research Reports

We study optimal control problems for hyperbolic equations (focusing on the multidimensional wave equation) with control functions in the Dirichlet boundary conditions under hard/pointwise control and state constraints. Imposing appropriate convexity assumptions on the cost integral functional, we establish the existence of optimal control and derive new necessary optimality conditions in the integral form of the Pontryagin Maximum Principle for hyperbolic state-constrained systems.