Weakly supervised brain tumor segmentation via semantic affinity deep neural network

Image segmentation tasks are considered resource intensive. These tasks require domain specialists to labor manually over long periods of time. When considering medical image segmentation tasks, the personnel and the error margin make these tasks expensive. Therefore there is a need for an automated tool. Deep learning has fast become the state of the art for such tasks, yet the methods applied require large data-sets of fully annotated examples. The need for supervision prevents researchers from developing deep learning and machine learning solutions on new datasets, which were not annotated by professional personnel. In this paper we utilize weak supervision to train a deep neural network to perform instance segmentation. The data used for this project is the Multimodal Brain Tumor Segmentation Challenge 3D MRI scans. The method used is a two-step DNN. The first step is binary classification of slices to either pathological or healthy. This is the only step which uses supervision for the training of the DNNs. In the second step, another DNN in the form of a Unet encoder-decoder network is utilized. This network encodes the input raw data and decodes each pixel to a 32 dimensional vector representing a semantic identity (semantic map). The supervision for training this second network is derived from the GradCAM of the classification DNN. Lastly, to segment the input data we determine the semantic distance between suspected lesion points and the entirety of the map. We achieve an average Dice score of 0.73 over three test sets of 38 patients each.