Applied Mathematics Commons^™

On The Optimal Reward Function Of The Continuous Time Multiarmed Bandit Problem, José Luis Menaldi, Maurice Robin

Mathematics Faculty Research Publications

The optimal reward function associated with the so-called "multiarmed bandit problem" for general Markov-Feller processes is considered. It is shown that this optimal reward function has a simple expression (product form) in terms of individual stopping problems, without any smoothness properties of the optimal reward function neither for the global problem nor for the individual stopping problems. Some results relative to a related problem with switching cost are obtained.

Remarks On Estimates For The Green Function, Jose Luis Menaldi

Mathematics Faculty Research Publications

No abstract provided.

Some Estimates For Finite Difference Approximations, José-Luis Menaldi

Mathematics Faculty Research Publications

Some estimates for the approximation of optimal stochastic control problems by discrete time problems are obtained. In particular an estimate for the solutions of the continuous time versus the discrete time Hamilton-Jacobi-Bellman equations is given. The technique used is more analytic than probabilistic.

On Asymptotic Behavior Of Stopping Time Problems, Jose Luis Menaldi, Maurice Robin

Mathematics Faculty Research Publications

No abstract provided.

On The Numerical Approximations Of An Optimal Correction Problem, M. C. Bancora-Imbert, P. L. Chow, J. L. Menaldi

Mathematics Faculty Research Publications

The numerical solution of an optimal correction problem for a damped random linear oscillator is studied. A numerical algorithm for the discretized system of the associated dynamic programming equation is given. To initiate the computation, we adopt a numerical scheme derived from the deterministic version of the problem. Next, a correction-type algorithm based on a discrete maximum principle is introduced to ensure the convergence of the iteration procedure.

Additive Control Of Stochastic Linear Systems With Finite Horizon, Pao-Liu Chow, José-Luis Menaldi, Maurice Robin

Mathematics Faculty Research Publications

We consider a dynamic system whose state is governed by a linear stochastic differential equation with time-dependent coefficients. The control acts additively on the state of the system. Our objective is to minimize an integral cost which depends upon the evolution of the state and the total variation of the control process. It is proved that the optimal cost is the unique solution of an appropriate free boundary problem in a space-time domain. By using some decomposition arguments, the problems of a two-sided control, i.e. optimal corrections, and the case with constraints on the resources, i.e. finite fuel, can be …

Optimal Stochastic Scheduling Of Power Generation Systems With Scheduling Delays And Large Cost Differentials, G. L. Blankenship, J.-L. Menaldi

Mathematics Faculty Research Publications

The optimal scheduling or unit commitment of power generation systems to meet a random demand involves the solution of a class of dynamic programming inequalities for the optimal cost and control law. We study the behavior of this optimality system in terms of two parameters: (i) a scheduling delay, e.g., the startup time of a generation unit; and (ii) the relative magnitudes of the costs (operating or starting) of different units. In the first case we show that under reasonable assumptions the optimality system has a solution for all values of the delay, and, as the delay approaches zero, that …

Optimal Control Of Stochastic Integrals And Hamilton-Jacobi-Bellman Equations, Ii, Pierre-Louis Lions, José-Luis Menaldi

Mathematics Faculty Research Publications

We consider the solution of a stochastic integral control problem, and we study its regularity. In particular, we characterize the optimal cost as the maximum solution of ∀v ∈ V, A(v)u ≤ ƒ(v) in D'(Ο), u = 0 on ∂Ο, u ∈ W^1,∞(Ο),

where A(v) is a uniformly elliptic second order operator and V is the set of the values of the control.

Optimal Control Of Stochastic Integrals And Hamilton-Jacobi-Bellman Equations, I, Pierre-Louis Lions, José-Luis Menaldi

Mathematics Faculty Research Publications

We consider the solution of a stochastic integral control problem and we study its regularity. In particular, we characterize the optimal cost as the maximum solution of ∀v ∈ V, A(v)u ≤ ƒ(v) in D'(Ο), u = 0 on ∂Ο, u ∈ W^1,∞(Ο),

where A(v) is a uniformly elliptic second order operator and V is the set of the values of the control.

On The Optimal Stopping Time Problem For Degenerate Diffusions, J. L. Menaldi

Mathematics Faculty Research Publications

In this paper we give a characterization of the optimal cost of a stopping time problem as the maximum solution of a variational inequality without coercivity. Some properties of continuity for the optimal cost are also given.

On The Optimal Impulse Control Problem For Degenerate Diffusions, J. L. Menaldi

Mathematics Faculty Research Publications

In this paper, we give a characterization of the optimal cost of an impulse control problem as the maximum solution of a quasi-variational inequality without assuming nondegeneracy. An estimate of the velocity of uniform convergence of the sequence of stopping time problems associated with the impulse control problem is given.