RF-DCM: Multi-Granularity Deep Convolutional Model Based on Feature Recalibration and Fusion for Driver Fatigue Detection

Fatigue driving is one of the main causes of traffic accidents. For real-world driver fatigue detection, the large pose deformations exhibited by the captured global face significantly increase the difficulty of extracting effective features. Furthermore, previous fatigue detection methods have not achieved desired results in distinguishing actions with similar appearance, such as yawning and speaking. In this article, we propose a multi-granularity Deep Convolutional Model based on feature Recalibration and Fusion for driver fatigue detection (RF-DCM). Our deep model leverages cues from partial faces to alleviate the pose variations and obtains robust feature representations from both the global face and different local parts. The core innovative techniques are as follows: A multi-granularity extraction sub-network extracts more efficient multi-granularity features while compressing the parameters of the network. In order to match multi-granularity features, a feature rectification sub-network and a feature fusion sub-network are designed to adaptively recalibrate and fuse the multi-granularity features. A long short term memory network is used to explore the relationship among sequence frames to distinguish actions with similar appearances. Extensive experimental results on the public drowsy driver dataset from NTHU Driver Drowsy competition demonstrate significant performance improvements of our model over all published state-of-the-art methods.