GSM=SSN: recurrent neural circuits optimised for probabilistic inference

ECHEVESTE, RODRIGO; HENNEQUIN, GUILLAUME; LENGYEL, MÁTÉ

Encuentro; Computational and Systems Neuroscience (Cosyne) Meeting 2017; 2017

The operation of cortical circuits has traditionally been studied using either bottom-up or top-down ap-proaches. The former identified dynamical mechanisms responsible for a wealth of empirical data, butwithout reference to computational function, while the latter extracted signatures of specific computationsfrom neural activity, but remained agnostic as to the underlying mechanisms. Here we bridge these twoapproaches and study the dynamics and function of cortical circuits in a principled unifying framework. Incontrast to recent optimisation-based approaches, which use highly simplified architectures and only ad-dress trial-average responses, here we train stochastic, recurrent neural circuits with realistic componentsthat allow us to link response dynamics and variability more directly to computational function. We trainnetworks to perform sampling-based probabilistic inference under a widely-used generative model of nat-ural images, the Gaussian Scale Mixture (GSM) model. We first show that the GSM posterior mean growswith stimulus contrast z, superlinearly for small z and saturating for large z, while the posterior variance de-creases with z. We then employ a novel, assumed density filtering-based approach to obtain the momentsof activity in stochastic networks as smooth, differentiable functions of network parameters, and matchthem to those of the GSM posterior for a set of training stimuli. We show that the network appropriatelygeneralizes to novel stimuli, reproducing the scaling of means and variances with contrast. Furthermore,the networks thus obtained operate in the dynamical regime of stabilised supralinear networks (SSN) thathas recently been proposed to underlie response normalization in V1. Thus, our results suggest a genericfunction for inhibition stabilised dynamics with a loose excitatory-inhibitory balance: they provide idealsubstrates of recognition models for probabilistic inference. Conversely, our approach could also be used toinfer the brain?s internal models based on observed dynamics.