A Deep Learning Approach For Hybrid Hand Gesture Recognition

DIEGO ALONSO; ALFREDO TEYSEYRE; LUIS BERDUN; SILVIA SCHIAFFINO

Xalapa, Veracruz, Mexico

Conferencia; 18th Mexican International Conference on Artificial Intelligence; 2019

Mexican Society for Artificial Intelligence (SMIA)

Emerging depth sensors and new interaction paradigms enable to create more immersive and intuitive Natural User Interfaces byrecognizing body gestures. One of the vision-based devices that has received plenty of attention is the Leap Motion Controller (LMC). Thisdevice models the 3D position of hands and fingers and provides morethan 50 features such as palm center and fingertips. In spite of the factthat the LMC provides such useful information of the hands, developersstill have to deal with the hand gesture recognition problem. For thisreason, several researchers approached this problem using well-knownmachine learning techniques used for gesture recognition such as SVMfor static gestures and DTW for dynamic gestures. At this point, wepropose an approach that applies a resampling technique based on fastFourier Transform algorithm to feed a CNN+LSTM neural network in order to identify both static and dynamic gestures. As far as our knowledge,there is no full dataset based on the LMC that includes both types ofgestures. Therefore, we also introduce the Hybrid Hand Gesture Recognition database, which consists of a large set of gestures generated withthe LMC, including both type of gestures with dierent temporal sizes.Experimental results showed the robustness of our approach in terms ofrecognizing both type of gestures. Moreover, our approach outperformsother well-known algorithms of the gesture recognition field.