Automated Personnel Digital Twinning in Industrial Workplaces

FERNANDEZ, MANLIO MASSIRIS; DELRIEUX, CLAUDIO; MUNOZ, J. ALVARO FERNANDEZ

International Conference on Electrical, Computer, and Energy Technologies, ICECET 2022

Institute of Electrical and Electronics Engineers Inc.

Año: 2022

Digital twin technology is designed to use real world data together with simulation programs to understand or predict how a process or activity is being performed. This technology arose in the last decade in several industrial contexts, mostly focused on devices, facilities and manufacturing, and only very recently to twinning human behavior. This paper describes an applied study on digital twinning of workers within complex and varying industrial settings, using the recently introduced VIBE neural network. VIBE comprises a two-stage pipeline of neural network blocks (NNB), where the first stage determines detection and tracking, while the second stage performs 2D-to-3D human model mapping. The first stage allows using of three different NNB combinations, namely Mask R-CNN & SORT, YOLO & SORT, and STAF. In this work, we focus on the influence of each NNB on the second stage and the final output using state-of-the-art performance metrics. Mask R-CNN & SORT provided a multi object tracking accuracy (MOTA) of 87,2%, while STAF obtained a MOTA 88,5%, and the best performance with occlusions in the final 2D-to-3D human rendering. These results pave the road for new developments in real-time complex digital twinning and workplace virtualization, in which human activities are an essential part.