Multimodal medical image fusion review: Theoretical background and recent advances

Multimodal medical image fusion consists in combining two or more images of the same or different modalities aiming to improve the image content, and preserve information. The rapid advance in medical imaging techniques (Computed Tomography (CT), Positron Emission Tomography (PET), Magnetic Resonance Imaging (MRI), Single Photon Emission Computed Tomography (SPECT)) has attracted researcher’s attention to fuse different modalities in order to assist experts decision making during the aided-diagnosis pipeline. Moreover, the fused results may help boosting other tasks such as classification, detection and segmentation. The main objective of this work is to provide a comprehensive overview of medical image fusion methods with theoretical background and recent advances. To do so, we present a detailed literature panorama of medical image fusion. The pixel-level, feature-level and decision-level fusion methods are highlighted and discussed with several approaches in each category. Theories behind fusion algorithms are explored aiming to address challenges and limitations of each classes. Therefore, we propose an experimental analysis of fusion performance given by different categories to guide the discussion. By summarizing the existing fusion classes, we discuss merits and demerits of each category to provide some recommendations for future research directions. Finally, performance evaluation metrics are presented to draw conclusions and perspectives.