Compressing a Neural Network Classifier using a Volterra-Neural Network model

M. RUBIOLO, G. STEGMAYER, D. MILONE

Barcelona

Conferencia; The 2010 International Joint Conference on Neural Networks (IJCNN); 2010

IEEE

Model compression is a required task when slowand large models are used, for example, for classification, butthere are transmissions, space, time or computing capabilitiesconstraints that have to be fulfilled. Multilayer Perceptron(MLP) models have been traditionally used as classifiers.Depending on the problem, they may need a large numberof parameters (neuron functions, weights and bias) to obtainan acceptable performance. This work proposes a techniqueto compress an MLP model preserving, at the same time, itsclassification performance, through the kernels of a Volterraseries model. The Volterra kernels can be used to represent theinformation that a Neural Network (NN) model has learnt withalmost the same accuracy but compressed into less parameters.The Volterra-NN approach proposed in this work has twoparts. First of all, it allows extracting the Volterra kernelsfrom the NN parameters after training, which will containthe classifier knowledge. Second, it allows building differentorders Volterra series model for the original problem using theVolterra kernels, significantly reducing the number of neuralparameters involved to a very few Volterra-NN parameters(kernels). Experimental results are presented over the standardIris classification problem, showing the good Volterra-NN modelcompression capabilities.