Decision Level Fusion for Audio-Visual Speech Recognition in Noisy conditions

GONZALO SAD; LUCAS TERISSI; JUAN CARLOS GÓMEZ

LECTURE NOTES IN COMPUTER SCIENCE

Springer

Año: 2017

0302-9743

This paper proposes a decision level fusion strategy for audiovisualspeech recognition in noisy situations. This method aims to enhancethe recognition over different noisy conditions by fusing the scoresobtained with classifiers trained with different feature sets. In particular,this method is evaluated by considering three modalities, audio,visual and audio-visual, respectively, but it could be employed using asmany modalities as needed. The combination of the scores is performedby taking into account the reliability of each modality at different noisyconditions. The performance of the proposed recognition system is evaluatedover two isolated word audio-visual databases, a public one and adatabase compiled by the authors of this paper. The proposed decisionlevel fusion strategy is evaluated by considering different kind of classifier.Experimental results show that a good performance is achieved withthe proposed system, leading to improvements in the recognition ratesthrough a wide range of signal-to-noise ratios.