Airborne SAR to Optical Image Registration Based on SAR Georeferencing and Deep Learning Approach

Synthetic aperture radar (SAR) to optical image registration is a crucial pre-processing step in remote sensing applications. As a multisource image registration problem, it has several challenges due to radiometric and geometric differences and the presence of speckle noise in SAR images. This article presents a coarse to fine registration approach based on georeferencing and a deep learning framework to deal with these problems. The purpose of the method is to combine georeferencing information and a deep learning registration approach to reduce the outliers and increase the ratio of correct correspondences (ROCC). In the georeferencing approach, using the geometry of the SAR payload, latitude and longitude are assigned to each pixel of the SAR image. It is, therefore, possible to make an initial match between the SAR and a georeferenced optical image. It should be noted that due to the inherent errors of georeferencing, the image-based approach as a fine registration step is inevitable. In the training phase, the SAR-SIFT and scale-invariant feature transform (SIFT) algorithms are applied to pairs of registered SAR and optical images, respectively. If the detected keypoints in the two images are spatially matched, the descriptors are applied to a deep neural network (DNN). The network is trained to create a binary output for the corresponding and noncorresponding descriptors. In the validation stage, using the trained network and the georeferencing, the number of incorrect correspondences can be effectively reduced. The experimental results on several pairs of SAR and optical modalities indicate the effectiveness of the proposed algorithm in terms of registration accuracy and robustness.