Pyramid-attentive GAN for multimodal brain image complementation in Alzheimer’s disease classification

Multimodal medical imaging has a larger volume of data compared to unimodal medical imaging, and can reflect different biological information and tissue features of the human body, complementing the structural details and image features missing from unimodal images to achieve a more accurate and comprehensive classification and diagnosis of diseases. In Alzheimer’s disease, the PET scan imaging technique is difficult to operate and expensive to detect, and is not included in routine examinations, resulting in a lack of PET image data in the dataset. In this paper, we propose a Generative Adversarial Network based on pyramidal attention mechanism to generate PET images through pyramidal attention mechanism and standard discriminators, which can effectively solve the problem of lack of PET data, complete the multimodal data sets of MRI and PET, combine the grey matter part of MRI images with the metabolic information in PET images to achieve multimodal medical image information fusion, and achieve classification and diagnosis of fused images through neural network. The experimental results of AD:MCI:NC triple classification show that our method achieves 89.9% accuracy.