Feature selection with simple ANN ensembles

J. IZETTA; P. M. GRANITTO

Jujuy, Argentina

Congreso; XV Congreso Argentino de Ciencias de la Computación - CACIC 2009; 2009

Red UNCI

Feature selection is a well-known pre-processing technique, commonly used with high-dimensional datasets. Its main goal is to discard useless or redundant variables, reducing the dimensionality of the input space, in order to increase the performance and interpretability of models. In this work we introduce the ANN-RFE, a new technique for feature selection that combines the accurate and time-efficient RFE method with the strong discrimination capabilities of ANN ensembles. In particular, we discuss two feature importance metrics that can be used with ANN-RFE: the shuffling and $dE$ metrics. We evaluate the new method using an artificial example and five real-world wide datasets, including gene-expression data. Our results suggest that both metrics have equivalent capabilities for the selection of informative variables. ANN-RFE seems to produce overall results that are equivalent to previous efficient methods, but can be more accurate on particular datasets.