From Stationarity to ICA: an Objective Function for Hebbian self-stabilizing Plasticity Rules.

ECHEVESTE, RODRIGO; ECKMANN, SAMUEL; GROS, CLAUDIUS

Congreso; Osnabrück Computational Cognition Alliance Meeting (Occam) 2015; 2015

Objective functions provide a useful framework for the formulation of guiding principles indynamical systems. In the case of information processing systems, such as neural networks, theseguiding principles can be formulated in terms of information theoretical measures with respect tothe input and output probability distributions. In the present work, a guiding principle for neuralplasticity is formulated in terms of an objective function defined as the Fisher information withrespect to an operator that we denote as the synaptic flux [1] . By minimization of this objectivefunction, we obtain synaptic plasticity rules that both account for Hebbian/anti-Hebbian learningand are self-limiting to avoid unbounded weight growth.The simplicity of the rules, taking a polynomial shape in the membrane potential of theneuron, allows us to study the attractors and stability of the learning procedure, and to express thesein terms of the cumulative moments of the input distribution. A preference for non-Gausiann inputdirections is found, making the rules suitable for independent component analysis (ICA).As an application, the non-linear bars problem [2] is studied, in which each neuron ispresented with a grid of LxL inputs, depicting a random set of bars. Each pixel can take two values;one for low intensity and one for high intensity, with a bar consisting of a complete row or acomplete column of high intensity pixels. At the intersection of a horizontal and vertical bar, theintensity is the same as if only one bar were present, which makes the problem non-linear. We showthat, under the here presented rules, the neurons are able to learn single bars (the independentcomponents of the input), even when these are never presented in isolation.