A Coarse-to-Fine 3D U-Net Network for Semantic Segmentation of Kidney CT Scans

The number of kidney cancer patients is increasing each year. Computed Tomography (CT) scans of the kidneys are useful to assess tumors and study tumor morphology. Semantic segmentation techniques enable the identification of kidney and surrounding anatomy on the pixel level. This allows clinicians to provide accurate treatment plans and improve efficiency. The large size of CT volumes poses challenges for deep segmentation methods as it cannot be accommodated on a single GPU in its original resolution. Downsampling CT scans influences the segmentation performance. In this paper, we present a coarse-to-fine cascaded network based on 3D U-Net architecture for semantic segmentation of kidney CT volumes into three classes kidney, tumor, and cyst. A two stage approach is implemented where a 3D U-Net model is first trained on downsampled CT volumes to delineate kidney region. This is followed by another 3D U-Net model which is trained using the full resolution images cropped around the areas of interest generated by first stage segmentation results. A set of 300 CT scans were used for training and evaluation. The proposed approach scored 0.9748, 0.8813, 0.8710 average dice for kidney, tumor and cyst using 3D cascade U-Net model. The performance of the cascade network outperformed other trained U-Net models based on 2D, 3D low resolution and 3D full resolution. The model also achieved the $$3^{rd}$$ place in the leaderboard of KiTS21 challenge with a mean sampled average dice score of 0.8944 and a mean sampled average surface dice score of 0.8140 using a test set of 100 CT scans.