Random Forest-like strategies for Neural Network Ensembles Construction

R. NAMÍAS; P. M. GRANITTO

Resistencia/Corrientes Argentina

Congreso; XIII Congreso Argentino de Ciencias de la Computación - CACIC 2007; 2007

Ensemble methods show improved generalization capabilities that outperform those of single learners. It is generally accepted that, for aggregation to be effective, the individual learners must be as accurate and diverse as possible. An important problem in ensemble learning is then how to find a good balance between these two conflicting conditions. For tree-based methods a successful strategy was introduced by Breiman with the Random-Forest algorithm. In this work we introduce new methods for neural network ensemble construction that follow Random-Forest-like strategies to construct ensembles. Using several real and artificial regression problems, we compare our new methods with the more typical Bagging algorithm and with three state-of-the-art regression methods. We find that our algorithms produce very good results on several datasets. Some evidence suggest that our new methods work better on problems with several redundant or noisy inputs.