Neuronal mechanisms underlying exploration-exploitation strategies in operant learning.

LEW, S.; REY, H.G.; SILVANO ZANUTTO, B.

NEURAL NETWORKS

PERGAMON-ELSEVIER SCIENCE LTD

Lugar: Amsterdam; Año: 2013 p. 1 - 6

0893-6080

Neuronal mechanisms underlying exploration-exploitation strategies in operant learning Abstract? One of the most valuable mechanisms in animal self-adaptation is the ability to switch between exploration and exploitation strategies. In this work, we present a computational model that learns visual discrimination paradigms and adapts its behavior whereupon rules change. In the model, dopamine and norepinephrine neurons are proposed as detectors of changes in the environment. Dopamine modulates the excitability and plasticity of artificial neurons in the prefrontal cortex and motor-related structures. These neurons change their synaptic weights following a Hebbian or anti-Hebbian rule depending on the amount of released dopamine and, as the reward rate increases, it induces exploitative behaviors. On the other hand, tonic levels of norepinephrine modulate both, information flows towards motor structures and the excitability of dopaminergic neurons, facilitating the switch from exploitation to exploration strategies. The computational model predicts behavioral and physiological results and provides a computational framework to the exploration-exploitation dilemma in self-adaptive agents.