Deformable histopathology-MRI image registration using deep learning

It is important to correlate histopathology images with radiological images to establish disease signatures and radiomic features defined on radiological images. Although modern imaging modalities such as CT and MRI can provide detailed patient anatomy information, the pathological interpretation of these images may be difficult and subject to physician's experiences. In contrast, histopathology images which provide tissue structure at cellular level is considered the gold standard for cancer diagnosis. Therefore, histopathological images fusion with radiological images helps interpretation and computer-aided disease classification and segmentation. In this study, we propose a new MR volume to histopathology slice image registration method using unsupervised deep learning (DL). The pathology images were first manually rotated to align with the MRI. Histopathology slice correspondence to the MRI slice was then established by calculating image similarity between the two. Because of the endorectal MRI coil, the MRI prostate is often pressed, causing deformation, from the posterior side. Even without endorectal coil, the bladder could cause the shape mismatch between the histology and MRI. Therefore, it is important to model the prostate deformation. In this study, a thin-plate-spline deformation model was calculated from prostate surface difference between the MRI slice and the histopathology. To match the image content, a DL-based method was proposed to deformably register the histopathology image to the MRI. The proposed method utilizes modality independent neighborhood descriptor (MIND) as the image similarity measure during network training. Tested on 10 cases, the SSIM between the two were on average 0.83 and 0.90 before and after registration. Visual inspection suggests good image registration performance of the proposed method.