How to train your dRagoNN: teaching neural networks probabilistic inference under biological constraints

ECHEVESTE, RODRIGO; HENNEQUIN, GUILLAUME; LENGYEL, MÁTÉ

Conferencia; Bernstein Conference 2017; 2017

The study of the dynamics and function of cortical circuits has typically been done either in a bottom-up fashion, identifying the biological mechanisms responsible for a wealth of empirical data, without reference to computational function, or by top-down approaches which link features of neural activity to specific computations, without committing to a particular implementation in the brain.  Here we bridge these two approaches and study the dynamics and function of cortical circuits in a principled unifying framework. We do so by training stochastic, recurrent neural circuits (Fig. 1 A in red) with realistic components (using experimentally observed non-linear dynamics and respecting Dale´s principle), in a way that allows us to establish a direct link between response dynamics and variability to computational function. We train these networks to perform sampling-based probabilistic inference under a simple though powerful and widely-used generative model of natural images, the Gaussian Scale Mixture (GSM) model [Wainright & Simoncelli, 2000] (Fig. 1 A in green) .  We first show that the GSM posterior mean grows with stimulus contrast $z$, superlinearly for small $z$ and saturating for large $z$, while the posterior variance decreases with $z$, in line with the findings of [Orbán et al., 2016] (Fig. 1 B in green). We then employ a novel, assumed density filtering-based approach [Hennequin & Lengyel, 2016] to obtain the moments of activity in stochastic networks as smooth, differentiable functions of network parameters, and match them to those of the GSM posterior for a set of training stimuli (Fig. 1 B in red). This allows us to train the network to reproduce not only mean responses to a set of stimuli, but also capture the effect on neural variability. Furthermore, we show that the network appropriately generalizes to novel stimuli (Fig. 1 B left column), reproducing the scaling of means and variances with contrast. Finally, we show how the network we obtain in this way operates in the dynamical regime of stabilised supralinear networks (SSN) that has recently been proposed to underlie response normalization in V1 [Rubin et al., 2015].  Thus, our results suggest a generic function for inhibition stabilised dynamics with a loose excitatory-inhibitory balance: they provide ideal substrates of recognition models for probabilistic inference. Conversely, our approach could also be used to infer the brain´s internal models based on observed dynamics.