Asymmetric 3D face model for Speech Language Pathologist applications

GONZALO DANIEL SAD; FACUNDO REYES; JULIÁN ALVAREZ

Jodhpur

Conferencia; 16th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2021); 2021

A novel method for facial and gesture analysis aiming to quantify different subjective measures employed in the Speech Language Pathologist area is proposed in this paper. The developed system can track the facial movements and expressions from an input video in order to extract useful morphological and gestural parameters that are of interest in different fields of study, such as, Speech Language Pathologist, Neurology, etc. A modified version of a 3D face model, named Candide-3, is employed in the tracking stage. Since the original 3D model cannot handle asymmetrical facial movements, a new set of animation units was implemented in order to effectively track asymmetrical gestures. To enhance the tracking accuracy, a fusion scheme is proposed in the facial gesture tracking stage by means of the combination of the 3D face model previously described and facial landmarks detected using deep learning models. Several perceptual experiments were carried out, achieving promising results.