Isolated Word Speech Recognition using AV Features

GONZALO SAD

Santiago

Taller; IEEE RAS Summer School on "Robot Vision and Applications"; 2012

School of Engineering of the Universidad de Chile

In this paper, the improvements on speech recognition rates by the inclusion of visual data related to mouth movements are presented. A Speech Recognition System combining Audio-Visual Hidden Markov Models to represent the correlation between the acoustic signal and facial movements during speech, and a visual extraction technique based on a simple 3D model, is proposed in this work. A Spanish digits dataset is employed to test the proposed system. The experimental results shown that a significant improvement is achieved when the visual information is considered. This is more noticeable for the case of low SNR.