HI2D2FNet: Hyperspectral Intrinsic Image Decomposition Guided Data Fusion Network for Hyperspectral and LiDAR Classification

In multimodal data fusion and land-cover interpretation tasks, the fusion interpretability between hyperspectral image (HSI) and light detection and ranging (LiDAR) data is always nontrivial to be clarified. Furthermore, the heterogeneous sample and distribution variances of these two remote sensing (RS) modalities impede the joint classification performance. In this paper, a Hyperspectral Intrinsic Image Decomposition guided Data Fusion Network (HI ² D ² FNet) is proposed. Generally, classic hyperspectral intrinsic image decomposition (HIID) performs well in image enhancement and shadow removal. It decomposes one HSI into one reflectance component and one shading component. Inspired by the core mechanism of HIID, our motivation is to preliminarily exploit its potential for multimodal RS data fusion and explore the inherent modality connection between HSI and LiDAR data from the intrinsic perspective. Specifically, compared with existing techniques, HI ² D ² FNet is capable of fusing the horizontal geometry information in the shading component with the vertical geometry information in the LiDAR data from both sample and distribution perspectives in the spatial domain. The generated cross-modal geometry feature contributes to guiding the reflectance stream optimization. This unique fusion framework connects both modalities with respect to geometry information and enhances the specific fusion interpretability. The decomposition and fusion modules in HI ² D ² FNet are optimized simultaneously in a novel alternative optimization pattern. Furthermore, several unique cross-modal constraints in terms of prior RS properties are presented. Experiments conducted on three widely available datasets prove the superiority of HI ² D ² FNet over state-of-the-art techniques. The source codes will be available at https://github.com/GEOywb/HI2D2FNet.