PPI Edge Infused Spatial–Spectral Adaptive Residual Network for Multispectral Filter Array Image Demosaicing

Multispectral filter array (MSFA) sensors provide a cost-effective and one-shot acquisition solution to obtain well-aligned multi-band images, which are helpful for various optical and remote sensing applications. However, the sparse spatial sampling rate and strong spectral cross-correlation make MSFA image demosaicing a challenging problem. Therefore, it is essential to develop effective MSFA demosaicing solutions to reconstruct full-resolution and high-fidelity multispectral images from the raw mosaic image. In this paper, we present a Pseudo-panchromatic Image (PPI) Edge infused Spatial-Spectral Adaptive Residual Network (PPIE-SSARN) for multispectral filter array image demosaicing. The proposed two-branch model deploys a residual sub-branch to adaptively compensate for the spatial and spectral differences of reconstructed multispectral images and a PPI edge infusion sub-branch to enrich the edge-related information. Moreover, we design an effective mosaic initial feature extraction module with a spatial- and spectral-adaptive weight-sharing strategy whose kernel weights can change adaptively with spatial locations and spectral bands to avoid artifacts and aliasing problems. Experimental results demonstrate the superiority of our proposed method, outperforming the state-of-the-art MSFA demosaicing approaches and achieving satisfying demosaicing results in terms of spatial accuracy and spectral fidelity. Our models and code will be publicly available.