Information theory approach to learning of the perceptron rule

L. DIAMBRA; J. FERNANDEZ

Physical Review E

AMERICAN PHYSICAL SOCIETY

Año: 2001 vol. 64 p. 461061 - 461068

By recourse to a method based on information theory, we have studied the generalization problem in perceptrons. We considered different a priori distributions about the weights of the teacher perceptron. Our approach allows us to define the information gain from the examples used in the training procedure. The information gain can be used to choose a convenient example set for training the perceptron and to select the transfer function of the student perceptron.