Visual Recognition of traffic police gestures with convolutional pose machine and handcrafted features

Abstract Autonomous vehicles have become a hot spot of the automotive industry, many cities have claimed that autonomous vehicles should be capable of recognizing gestures used by traffic police. Traditional traffic police gesture recognition methods rely on depth-sensor or wearable-devices, which limits their availability in the domain of the intelligent vehicle. Vision-based methods have fewer requirements for distance, but the modeling process is challenging due to the complexity of the visual scenes. Inspired by the recent success in vision-based pose estimation networks such as Convolutional Pose Machine (CPM), in this paper, we propose a novel vision-based human-machine interface to recognize eight kinds of Chinese traffic police gestures and apply it in the real-time recognition tasks. This method integrates a modified CPM network and two kinds of handcrafted features: Relative Bone Length and Angle with Gravity as spatial domain features, and adopt a Long short-term memory (LSTM) network to extract temporal domain features. To train and validate our method, we create a gestures dataset with two hours of traffic police gesture videos, which has 3354 gesture instances. The experiment results show that the proposed method is capable of recognizing traffic police gestures, and is fast enough for online gesture prediction.