SAR–Optical Image Matching With Semantic Position Probability Distribution

We propose a deep learning framework of Semantic Position Probability Distribution for SAR-optical image matching, termed as SPPD. Unlike the pixel-by-pixel searching matching method, a correspondence is directly obtained by an outputted matching position probability distribution. First, multiscale pyramidal features are created for each pixel in the SAR and optical images by using two weight-sharing ResNet-50 + Feature Pyramid Network (FPN) networks. The features containing high-level semantic information are then embedded into the proposed image Position Attention Module to obtain the spatial position dependencies between two images. Then, we present a loss function for semantic position matching to optimize the network from both semantic information and pixel alignment perspectives, converting the probability distribution of semantic matching positions into a point-to-point matching problem. In this paper, the SAR and optical images are set as the sensed and reference images. The effects of different image sizes, training label types, and loss function weights on matching accuracy are explored to obtain the optimal parameter settings for matching. The experimental results show that the proposed method is insensitive to image deformation and achieves cross-modal matching for SAR-optical images with high accuracy compared with the best matching method on different scene images, with several orders of magnitude faster inferences time.