A Traffic Surveillance System for Obtaining Comprehensive Information of the Passing Vehicles Based on Instance Segmentation

The vehicle information acquired by current video-based traffic surveillance methods is usually onefold. In this paper, a traffic surveillance system for obtaining comprehensive vehicle information, including type, number of axles, 3D bounding box, speed, length, current driving lane, and traffic volume, is proposed based on instance segmentation which is realized by Mask R-CNN. The work mainly contains: 1) An annotated image dataset which contains car, coach, truck, and wheel is established for training Mask R-CNN; 2) The method for identifying the number of axles based on MaskIoU is proposed; 3) The 3D bounding box is generated based on the high-quality vehicle segmentation, which is more accurate than that generated by background subtraction methods; 4) For calculating the vehicle speed and length, the reference points on the road plane are determined by the lane with dashed line and one vanishing point to calculate the homography matrix; 5) In order to keep the obtained vehicle information reliability, the results identified from multi-frames are obtained by means of the tracking method SORT for processing. The proposed system is verified in different scenes. The average recognition accuracies of the vehicle types and number of axles are more than 97% and 88% in three testing videos. The speed errors of more than 90% vehicles are less than 4%. And the ratios of both missing and repetition in vehicle counting are low in general.