Video-Based Windshield Rain Detection and Wiper Control Using Holistic-View Deep Learning

A windshield rain detection system using holistic-view deep learning of more than 150k real images is proposed in this article. A wiper control algorithm based on a time-series treatment is also presented. The training images were solicited from internet videos filmed by vehicles running in various weather conditions, including heavy rain and sunny day. The images were manually judged and divided into two categories recommending wiper activation or not. The labeled images were then used to train a deep convolutional neural network for wiper activation classification. A time-series control scheme was adopted to examine the sequence result of rain detection for commanding wiper activation in real time. Training for the day and the night scenes was conducted first individually and then jointly; the influence of the training data size was also tested and discussed. Overall, we achieved an average precision rate of 0.88 in our video-based rain detection experiments; our recall rate of 0.87 is significantly higher than the state-of-the-arts that averaged around 0.6. The time-series control scheme further increased the recall on correct timing of wiper activation by 5%. It is proved that the proposed system is practical for real-time vehicle windshield rain detection and wiper control.