Behavior monitoring under uncertainty using Bayesian surprise and optimal action selection

LUIS AVILA; E. C. MARTÍNEZ

EXPERT SYSTEMS WITH APPLICATIONS

PERGAMON-ELSEVIER SCIENCE LTD

Año: 2014 vol. 41 p. 6327 - 6345

0957-4174

The increasing trend towards delegating tasks to autonomous artificial agents in safety?critical sociotechnical systems makes monitoring an action selection policy of paramount importance. Agent behavior monitoring may profit from a stochastic specification of an optimal policy under uncertainty. A probabilistic monitoring approach is proposed to assess if an agent behavior (or policy) respects its specification. The desired policy is modeled by a prior istribution for state transitions in an optimally-controlled stochastic process. Bayesian surprise is defined as the Kullback?Leibler divergence between the state transition distribution for the observed behavior and the distribution for optimal action selection. To provide a sensitive on-line estimation of Bayesian surprise with small samples twin Gaussian processes are used. Timely detection of a deviant behavior or anomaly in an artificial pancreas highlights the sensitivity of Bayesian surprise to a meaningful discrepancy regarding the stochastic optimal policy when there exist excessive glycemic variability, sensor errors, controller ill-tuning and infusion pump malfunctioning. To reject outliers and leave out redundant information, on-line sparsification of data streams is proposed.