Progressive matching method of aerial-ground remote sensing image via multi-scale context feature coding

ABSTRACTThe fine 3D model is the essential spatial information for the construction of a smart city. UAV aerial images with large-scale scene perception ability are common data sources for 3D modelling of cities at present. However, in some complex urban areas, a single aerial image is difficult to capture the 3D scene information because of the existence of some problems such as inaccurate edges, holes, and blurred building facade textures due to changes in perspective and area occlusion. Therefore, how to solve perspective changes and area occlusion of the aerial image quickly and efficiently has become an important problem. The ground image can be used as an important supplement to solve the problem of missing bottom and area occlusion in oblique photography modelling. Thus, this article proposes a progressive matching method via multi-scale context feature coding network to achieve robust matching of aerial-ground remote sensing images, which provides better technical support for urban modelling. The main idea consists of three parts: (1) a multi-scale context feature coding network is designed to extract feature on aerial-ground images efficiently; (2) a block-based matching strategy is proposed to pay more attention to local features of the aerial-ground images; (3) a progressive matching method is applied in block matching stage to obtain more accurate features. We used eight sets of typical data, such as aerial images captured by the drone DJI-MAVIC2 and ground images captured by handheld devices as experimental objects, and compared them with algorithms such as SIFT, D2-net, DFM and SuperGlue. Experimental results show that our proposed aerial-ground image matching method has a good performance that the average NCM has improved 2.1–8.2 times, and the average rate of correct matching has an average increase of 26% points with the average root of mean square error is only 1.48 pixels.KEYWORDS: 3D modelaerial-ground remote sensing imagelarge buildingsfeature matchingdeep learning Disclosure statementNo potential conflict of interest was reported by the author(s).Additional informationFundingThe work was supported by the National Innovation and Entrepreneurship Training Project for University (China) [202210500028].