Semantic Segmentation of Traffic Landmarks Using Classical Computer Vision and U-Net Model

Abstract Due to the deterioration of traffic landmarks such as (crosswalks, lanes, speed limits, …etc) in Mie prefecture-Japan, the sudden stop rate during road crossings reached 3.4% in the year 2020. It has been deemed necessary by the local authorities to implement a reliable maintenance process for these road signs. Such a process would rely on an automatic camera-based monitoring system in order to keep track of the deterioration status of road signs. The first step consists of achieving successful detection and segmentation of these landmarks. In this paper, we try to accomplish this initial goal by using a dataset of around 13000 high resolution images taken from Mie prefecture streets. In order to achieve our goal of landmark detection we relied on some classical computer vision techniques as well as a deep learning approach. We developed a method to separate landmarks from other objects in a given image by relying on distinctive features of these landmarks. The method in question was then used to build a semi-assisted labelling tool, which was used to annotate a set of around 182 images and used them to train a CNN model to perform semantic segmentation of the road signs. In the end we managed to achieve a successful segmentation with a dice score of 78.90% on the validation set, as well as a solid proof of concept for a CNN based approach to solve this particular problem