Performance Variability in Zero-Shot Classification

MATIAS MOLINA; JORGE SANCHEZ

Workshop; NeurIPS LXAI Research Workshop; 2019

Zero-shot classification (ZSC) is the task of learning predictors for classes not seen during training. Although the different methods in the literature are evaluated using the same class splits, little is known about their stability under different class partitions. In this work we show experimentally that ZSC performance exhibits strong variability under changing training setups. We propose the use ensemble learning as an attempt to mitigate this phenomena. Experiments on four different datasets indicate that further investigations are necessary.