DriverMHG: A Multi-Modal Dataset for Dynamic Recognition of Driver Micro Hand Gestures and a Real-Time Recognition Framework

Koepueklue, Okan; Ledwon, Thomas; Rong, Yao; Kose, Neslihan und Rigoll, Gerhard (2020): DriverMHG: A Multi-Modal Dataset for Dynamic Recognition of Driver Micro Hand Gestures and a Real-Time Recognition Framework. In: 2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (Fg 2020): S. 77-84

Abstract

The use of hand gestures provides a natural alternative to cumbersome Interface devices for Human-Computer Interaction (HCl) Systems. However, real-time recognition of dynamic micro hand gestures from video streams is challenging for in-vehicle scenarios since (i) the gestures should be per-formed naturally without distracting the driver, (ii) micro hand gestures occur within very short time intervals at spatially con-strained areas, (iii) the performed gesture should be recognized only once, and (iv) the entire architecture should be designed lightweight as it will be deployed to an embedded System. In this work, we propose an HCl System for dynamic recognition of driver micro hand gestures, which can have a crucial impact in automotive sector especially for safety related issues. For this purpose, we initially coilected a dataset named Driver Micro Hand Gestures (DriverMHG), which consists of RGB, depth and infrared modalities. The challenges for dynamic recognition of micro hand gestures have been addressed by proposing a lightweight convolutional neural network (CNN) based architecture which operates online efiiciently with a sliding window approach. For the CNN model, several 3dimensional resource efficient networks are applied and their performances are analyzed. Online recognition of gestures has been performed with 3D-MobileNetV2, which provided the best offline accuracy among the applied networks with similar computational complexities. The final architecture is deployed on a driver Simulator operating in real-time. We make DriverMHG dataset and our source code publicly available(1).

• Dokumententyp: Zeitschriftenartikel
• Fakultät: Mathematik, Informatik und Statistik > Informatik
• Themengebiete: 000 Informatik, Informationswissenschaft, allgemeine Werke > 004 Informatik
• ISSN: 2326-5396
• Sprache: Englisch