Path integral control problems and Multilevel Monte Carlo method

SILVIA A. MENCHÓN; H. J. KAPPEN

Congreso; MACI VI; 2017

The main aim of stochastic optimal control theory is to compute an optimal sequence of actions to attain a future goal. When the system dynamics is subject to white Gaussian noise, it is possible to define a class of non-linear stochastic control problems that can be efficiently solved. Path integral control problems represent a restricted class of non-linear control problems with arbitrary dynamics and state cost, but with a linear dependence of the control on the dynamics and quadratic control cost. Since the path integral involves an expectation value with respect to a dynamical system, the optimal control can be estimated by implementing Monte Carlo sampling. Although importance sampling is used to improve numerical computations, the effective sample size may still be low. Here, we propose a way of implementing importance sampling with multilevel Monte Carlo and test it in a finite horizon control problem based on Lorenz-96 model.