Physical Sciences and Mathematics Commons^™

Optimal Stopping Problems For A Family Of Continuous-Time Markov Processes, Héctor Jasso-Fuentes, Jose-Luis Menaldi, Fidel Vásquez-Rojas

Mathematics Faculty Research Publications

In this paper we study the well-know optimal stopping problem applied to a general family of continuous-time Markov process. The approach to follow is merely analytic and it is based on the characterization of stopping problems through the study of a certain variational inequality; namely one solution of this inequality will coincide with the optimal value of the stopping problem. In addition, by means of this characterization, it is possible to find the so-named continuation region, and as a byproduct obtaining the optimal stopping time. The most of the material is based on the semigroup theory, infinitesimal generators and resolvents. …

On Some Optimal Stopping Problems With Constraint, J. L. Menaldi, M. Robin

Mathematics Faculty Research Publications

We consider the optimal stopping problem of a Markov process {x_t : t ≤ 0} when the controller is allowed to stop only at the arrival times of a signal, that is, at a sequence of instants {τ_n : n ≤ 1} independent of {x_t : t ≤ 0}. We solve in detail this problem for general Markov–Feller processes with compact state space when the interarrival times of the signal are independent identically distributed random variables. In addition, we discuss several extensions to other signals and to other cases of state spaces. These results …

Almost Sure Asymptotic Stabilization Of Differential Equations With Time-Varying Delay By Lévy Noise, Dezhi Liu, Weiqun Wang, Jose Luis Menaldi

Mathematics Faculty Research Publications

This paper aims to determine that the Lévy noise can stabilize the given differential equations with time-varying delay, which has generalized the Brownian motion case. An analysis is developed and sufficient conditions on the stabilization for stochastic differential equations with time-varying delay are presented. Our stabilization criteria is in terms of linear matrix inequalities (LMIs), whence the feedback controls can be designed more easily in practice.

On The Impulse Control Of Jump Diffusions, Erhan Bayraktar, Thomas Emmerling, José-Luis Menaldi

Mathematics Faculty Research Publications

Regularity of the impulse control problem for a nondegenerate n-dimensional jump diffusion with infinite activity and finite variation jumps was recently examined in [M. H. A. Davis, X. Guo, and G. Wu, SIAM J. Control Optim., 48 (2010), pp. 5276–5293]. Here we extend the analysis to include infinite activity and infinite variation jumps. More specifically, we show that the value function u of the impulse control problem satisfies u ∈ W_loc^2,p(Rⁿ).

Singular Ergodic Control For Multidimensional Gaussian-Poisson Processes, J. L. Menaldi, M. Robin

Mathematics Faculty Research Publications

Singular control for multidimensional Gaussian-Poisson processes with a long-run (or ergodic) and a discounted criteria are discussed. The dynamic programming yields the corresponding Hamilton-Jacobi-Bellman equations, which are discussed. Full details on the proofs and further extensions are left for coming works.

On The Lqg Theory With Bounded Control, D. V. Iourtchenko, J. L. Menaldi, A. S. Bratus

Mathematics Faculty Research Publications

We consider a stochastic optimal control problem in the whole space, where the corresponding HJB equation is degenerate, with a quadratic running cost and coeffcients with linear growth. In this paper we provide a full mathematical details on the key estimate relating the asymptotic behavior of the solution as the space variable goes to infinite.

Some Results Of Backward Itô Formula, Guiseppe Da Prato, Jose-Luis Menaldi, Luciano Tubaro

Mathematics Faculty Research Publications

We use the notion of backward integration, with respect to a general Lévy process, to treat, in a simpler and unifying way, various classical topics as: Girsanov theorem, rst order partial differential equations, the Liouville (or Lyapunov) equations and the stochastic characteristic method.

A Distributed Parabolic Control With Mixed Boundary Conditions, Jose-Luis Menaldi, Domingo Alberto Tarzia

Mathematics Faculty Research Publications

We study the asymptotic behavior of an optimal distributed control problem where the state is given by the heat equation with mixed boundary conditions. The parameter α intervenes in the Robin boundary condition and it represents the heat transfer coefficient on a portion Γ₁ of the boundary of a given regular n-dimensional domain. For each α, the distributed parabolic control problem optimizes the internal energy g. It is proven that the optimal control ĝ_α with optimal state u_ĝαα and optimal adjoint state p_ĝαα are convergent as α → 1 …

Remarks On Risk-Sensitive Control Problems, José Luis Menaldi, Maurice Robin

Mathematics Faculty Research Publications

The main purpose of this paper is to investigate the asymptotic behavior of the discounted risk-sensitive control problem for periodic diffusion processes when the discount factor α goes to zero. If u_α(θ, x) denotes the optimal cost function, being the risk factor, then it is shown that lim_α→0αu_α(θ, x) = ξ(θ) where ξ(θ) is the average on ]0, θ[ of the optimal cost of the (usual) in nite horizon risk-sensitive control problem.

Penalty Approximation And Analytical Characterization Of The Problem Of Super-Replication Under Portfolio Constraints, Alain Bensoussan, Nizar Touzi, José Luis Menaldi

Mathematics Faculty Research Publications

In this paper, we consider the problem of super-replication under portfolio constraints in a Markov framework. More specifically, we assume that the portfolio is restricted to lie in a convex subset, and we show that the super-replication value is the smallest function which lies above the Black-Scholes price function and which is stable for the so-called face lifting operator. A natural approach to this problem is the penalty approximation, which not only provides a constructive smooth approximation, but also a way to proceed analytically.

Impulse Control Of Stochastic Navier-Stokes Equations, J. L. Menaldi, S. S. Sritharan

Mathematics Faculty Research Publications

In this paper we study stopping time and impulse control problems for stochastic Navier-Stokes equation. Exploiting a local monotonicity property of the nonlinearity, we establish existence and uniqueness of strong solutions in two dimensions which gives a Markov-Feller process. The variational inequality associated with the stopping time problem and the quasi-variational inequality associated with the impulse control problem are resolved in a weak sense, using semigroup approach with a convergence uniform over path.

Stochastic Hybrid Control, A. Bensoussan, J. L. Menaldi

Mathematics Faculty Research Publications

The objective of this paper is to study the stochastic version of a previous paper of the authors, in which hybrid control for deterministic systems was considered. The modelling is quite similar to the deterministic case. We have a system whose state is composed of a continuous part and a discrete part. They are affected by a continuous type control and an impulse control. The dynamics is moreover perturbed by noise, also a continuous and a discrete noise process. The Markovian character of the state process is preserved. We develop the model and show how the dynamic programming approach leads …

Invariant Measure For Diffusions With Jumps, Jose-Luis Menaldi, Maurice Robin

Mathematics Faculty Research Publications

Our purpose is to study an ergodic linear equation associated to diffusion processes with jumps in the whole space. This integro-differential equation plays a fundamental role in ergodic control problems of second order Markov processes. The key result is to prove the existence and uniqueness of an invariant density function for a jump diffusion, whose lower order coefficients are only Borel measurable. Based on this invariant probability, existence and uniqueness (up to an additive constant) of solutions to the ergodic linear equation are established.

Ergodic Control Of Reflected Diffusions With Jumps, Jose-Luis Menaldi, Maurice Robin

Mathematics Faculty Research Publications

No abstract provided.

On An Investment-Consumption Model With Transaction Costs, Marianne Akian, José Luis Menaldi, Agnès Sulem

Mathematics Faculty Research Publications

This paper considers the optimal consumption and investment policy for an investor who has available one bank account paying a fixed interest rate and n risky assets whose prices are log-normal diffusions. We suppose that transactions between the assets incur a cost proportional to the size of the transaction. The problem is to maximize the total utility of consumption. Dynamic programming leads to a variational inequality for the value function. Existence and uniqueness of a viscosity solution are proved. The variational inequality is solved by using a numerical algorithm based on policies, iterations, and multigrid methods. Numerical results are displayed …

Optimal Starting-Stopping Problems For Markov-Feller Processes, Jose-Luis Menaldi, Maurice Robin, Min Sun

Mathematics Faculty Research Publications

By means of nested inequalities in semigroup form we give a characterization of the value functions of the starting-stopping problem for general Markov-Feller processes. Next, we consider two versions of constrained problems on the nal state or on the final time. The plan is as follows:

1. Introduction
2. Nested variational inequalities
3. Solution of optimal starting-stopping problem
4. Problems with constraints

References.

Optimal Control And Differential Games With Measures, E. N. Barron, R. Jensen, J. L. Menaldi

Mathematics Faculty Research Publications

We consider control problems with trajectories which involve ordinary measureable control functions and controls which are measures. The payoff involves a running cost in time and a running cost against the control measures. In the optimal control problem we are trying to minimize this payoff with both controls. In the differential game problem we are trying to minimize the cost with the ordinary controls assuming that the measure controls are chosen to maximize the cost. We will characterize the value functions in both cases using viscosity solution theory by deriving the Bellman and Isaacs equations.

Singular Ergodic Control For Multidimensional Gaussian Processes, J. L. Menaldi, M. Robin, M. I. Taksar

Mathematics Faculty Research Publications

A multidimensional Wiener process is controlled by an additive process of bounded variation. A convex nonnegative function measures the cost associated with the position of the state process, and the cost of controlling is proportional to the displacement induced. We minimize a limiting time-average expected (ergodic) criterion. Under reasonable assumptions, we prove that the optimal discounted cost converges to the optimal ergodic cost. Moreover, under some additional conditions there exists a convex Lipschitz continuous function solution to the corresponding Hamilton-Jacobi-Bellman equation which provides an optimal stationary feedback control.

On The Optimal Reward Function Of The Continuous Time Multiarmed Bandit Problem, José Luis Menaldi, Maurice Robin

Mathematics Faculty Research Publications

The optimal reward function associated with the so-called "multiarmed bandit problem" for general Markov-Feller processes is considered. It is shown that this optimal reward function has a simple expression (product form) in terms of individual stopping problems, without any smoothness properties of the optimal reward function neither for the global problem nor for the individual stopping problems. Some results relative to a related problem with switching cost are obtained.

Remarks On Estimates For The Green Function, Jose Luis Menaldi

Mathematics Faculty Research Publications

No abstract provided.

Some Estimates For Finite Difference Approximations, José-Luis Menaldi

Mathematics Faculty Research Publications

Some estimates for the approximation of optimal stochastic control problems by discrete time problems are obtained. In particular an estimate for the solutions of the continuous time versus the discrete time Hamilton-Jacobi-Bellman equations is given. The technique used is more analytic than probabilistic.

On Asymptotic Behavior Of Stopping Time Problems, Jose Luis Menaldi, Maurice Robin

Mathematics Faculty Research Publications

No abstract provided.

On The Numerical Approximations Of An Optimal Correction Problem, M. C. Bancora-Imbert, P. L. Chow, J. L. Menaldi

Mathematics Faculty Research Publications

The numerical solution of an optimal correction problem for a damped random linear oscillator is studied. A numerical algorithm for the discretized system of the associated dynamic programming equation is given. To initiate the computation, we adopt a numerical scheme derived from the deterministic version of the problem. Next, a correction-type algorithm based on a discrete maximum principle is introduced to ensure the convergence of the iteration procedure.

Optimal Stochastic Scheduling Of Power Generation Systems With Scheduling Delays And Large Cost Differentials, G. L. Blankenship, J.-L. Menaldi

Mathematics Faculty Research Publications

The optimal scheduling or unit commitment of power generation systems to meet a random demand involves the solution of a class of dynamic programming inequalities for the optimal cost and control law. We study the behavior of this optimality system in terms of two parameters: (i) a scheduling delay, e.g., the startup time of a generation unit; and (ii) the relative magnitudes of the costs (operating or starting) of different units. In the first case we show that under reasonable assumptions the optimality system has a solution for all values of the delay, and, as the delay approaches zero, that …

Optimal Control Of Stochastic Integrals And Hamilton-Jacobi-Bellman Equations, Ii, Pierre-Louis Lions, José-Luis Menaldi

Mathematics Faculty Research Publications

We consider the solution of a stochastic integral control problem, and we study its regularity. In particular, we characterize the optimal cost as the maximum solution of ∀v ∈ V, A(v)u ≤ ƒ(v) in D'(Ο), u = 0 on ∂Ο, u ∈ W^1,∞(Ο),

where A(v) is a uniformly elliptic second order operator and V is the set of the values of the control.

Optimal Control Of Stochastic Integrals And Hamilton-Jacobi-Bellman Equations, I, Pierre-Louis Lions, José-Luis Menaldi

Mathematics Faculty Research Publications

We consider the solution of a stochastic integral control problem and we study its regularity. In particular, we characterize the optimal cost as the maximum solution of ∀v ∈ V, A(v)u ≤ ƒ(v) in D'(Ο), u = 0 on ∂Ο, u ∈ W^1,∞(Ο),

where A(v) is a uniformly elliptic second order operator and V is the set of the values of the control.

On The Optimal Stopping Time Problem For Degenerate Diffusions, J. L. Menaldi

Mathematics Faculty Research Publications

In this paper we give a characterization of the optimal cost of a stopping time problem as the maximum solution of a variational inequality without coercivity. Some properties of continuity for the optimal cost are also given.

On The Optimal Impulse Control Problem For Degenerate Diffusions, J. L. Menaldi

Mathematics Faculty Research Publications

In this paper, we give a characterization of the optimal cost of an impulse control problem as the maximum solution of a quasi-variational inequality without assuming nondegeneracy. An estimate of the velocity of uniform convergence of the sequence of stopping time problems associated with the impulse control problem is given.