Polarization Image Demosaicing and RGB Image Enhancement for Color Polarization Sparse Focal Plane Array

The color division of focal plane (DoFP) polarization sensor structure mostly uses Bayer filter and polarization filter superimposed on each other, which makes the polarization imaging unsatisfactory in terms of photon transmission rate and information fidelity. In order to obtain high-resolution polarization images and high-quality RGB images simultaneously, we simulate a sparse division of focal plane polarization sensor structure, and seek a sweet spot of the simultaneous distribution of the Bayer filter and the polarization filters to obtain both high-resolution polarization images and high-quality RGB images. In addition, From the perspective of sparse polarization sensor imaging, leaving aside the traditional idea of polarization intensity interpolation, we propose a new sparse Stokes vector completion method, in which the network structure avoids the introduction and amplification of noise during polarization information acquisition by mapping the S1 and S2 components directly. The sparsely polarimetric image demosaicing (Sparse-PDM) model is a progressive combined structure of RGB image artifact removal enhancement network and sparsely polarimetric image completion network, which aims to compensate sparsely polarimetric Stokes parameter images with the de-artifacts RGB image as a guide, thus achieving high-quality polarization information and RGB image acquisition. Qualitative and quantitative experimental results on both self-constructed and publicly available datasets prove the superiority of our method over state-of-the-art methods.