Conats: A Novel Framework for Cross-Modal Map Extraction

Nowadays, the quality of digital maps is vital to various road-based applications, like autonomous driving, route recommendation, etc. The traditional way of map extraction/update through land surveying is expensive and usually fails to meet map recency requirements. The recent surge of automatic map extraction from GPS trajectory and/or aerial image data provides a cost-efficient way to update maps timely. However, extracting maps from solely GPS trajectories or aerial images can cause various map quality issues, and the latest neural network-based methods that fuse these two modalities do not consider their characteristics separately, so they suffer from the mutual perturbation of features. To address this issue, we propose a Cross-modal consistent enhancement and joint supervision framework (Conats) using both GPS trajectories and aerial images. It comprehensively extracts the local features and global features of each modality, then features of the same layer from different modalities are first fused to generate a modal consistent information gain which is used to enhance each modality’s features afterward. Moreover, we propose a new joint supervision prediction module that uses a combined loss function set consisting of Dice loss, focal loss, and BCE loss to better model the distinct features of different modalities. Extensive experiments are conducted on the Beijing and Porto datasets that show superior performance over existing works in terms of the accuracy of generated maps.