Traffic Sign Recognition Using a Multi-Task Convolutional Neural Network

Although traffic sign recognition has been studied for many years, most existing works are focused on the symbol-based traffic signs. This paper proposes a new data-driven system to recognize all categories of traffic signs, which include both symbol-based and text-based signs, in video sequences captured by a camera mounted on a car. The system consists of three stages, traffic sign regions of interest (ROIs) extraction, ROIs refinement and classification, and post-processing. Traffic sign ROIs from each frame are first extracted using maximally stable extremal regions on gray and normalized RGB channels. Then, they are refined and assigned to their detailed classes via the proposed multi-task convolutional neural network, which is trained with a large amount of data, including synthetic traffic signs and images labeled from street views. The post-processing finally combines the results in all frames to make a recognition decision. Experimental results have demonstrated the effectiveness of the proposed system.