POS-GIFT: A geometric and intensity-invariant feature transformation for multimodal images

Multimodal image matching suffers from severe geometric and nonlinear intensity distortion (NID). Towards this problem, we propose a multimodal image matching algorithm based on multi-orientation filtering results, called position-orientation-scale guided geometric and intensity-invariant feature transformation (POS-GIFT). First, we design a multi-layer circular point sampling pattern to effectively capture the local image structure. Then, we propose a novel feature descriptor that can work robustly across rotational differences in [0°, 360°) in the presence of NID. Specifically, we (1) integrate the multi-orientation filtering response in the local neighborhood with a Gaussian weight to form the feature of each sampled point (GFP), (2) build feature vectors for each orientation by concatenating the features of points grouped by orientation, (3) estimate the primary orientation by finding the feature vector with the largest norm which is constructed in the previous step, (4) modify the order of elements of GFP, and (5) finally concatenate the features of all sampled points in a certain order to form the complete feature descriptor. At last, we propose a position-orientation-scale guided inlier recovery strategy (POS) by integrating the global position, orientation, and scale information to further improve the matching performance, especially the number and distribution of correct matches in texture-less and complex areas. Experimental results on various multimodal datasets from remote sensing, medical, and computer vision imaging domains show that POS-GIFT outperforms eight state-of-the-art multimodal image feature matching algorithms which are five handcrafted-based methods, OS-SIFT, PSO-SIFT, LGHD, RIFT, and LNIFT, and three learning-based methods RedFeat, MatchFormer, and SemLA by several times in terms of correct matches while improving the root-mean-square error to around 1 pixel. Our implementation is available at https://github.com/Zhuolu-Hou/POS-GIFT.