A Deep Learning Approach for Hybrid Hand Gesture Recognition

DIEGO ALONSO; ALFREDO TEYSEYRE; LUIS BERDUN; SILVIA SCHIAFFINO

Conferencia; MICAI 2019, 18th Mexican International Conference on Artificial Intelligence; 2019

SMIA

Emerging depth sensors and new interaction paradigmsenable to create more immersive and intuitive Natural User Interfacesby recognizing body gestures. One of the vision-based devices that hasreceived plenty of attention is the Leap Motion Controller (LMC). Thisdevice models the 3D position of hands and fingers and provides morethan 50 features such as palm center and fingertips. In spite of the factthat the LMC provides such useful information of the hands, developersstill have to deal with the hand gesture recognition problem. For thisreason, several researchers approached this problem using well-knownmachine learning techniques used for gesture recognition such as SVM forstatic gestures and DTW for dynamic gestures. At this point, we proposean approach that applies a resampling technique based on fast FourierTransform algorithm to feed a CNN+LSTM neural network in orderto identify both static and dynamic gestures. As far as our knowledge,there is no full dataset based on the LMC that includes both types ofgestures. Therefore, we also introduce the Hybrid Hand Gesture Recognitiondatabase, which consists of a large set of gestures generated withthe LMC, including both type of gestures with different temporal sizes.Experimental results showed the robustness of our approach in terms ofrecognizing both type of gestures. Moreover, our approach outperformsother well-known algorithms of the gesture recognition field.