ABFNet: Attention Bottlenecks Fusion Network for Multimodal Brain Tumor Segmentation

Brain tumor segmentation based on multimodal magnetic resonance imaging (MRI) is a crucial step in the early diagnosis and prognosis evaluation of glioma. Current multimodal brain tumor segmentation algorithms only perform simple concatenation of multimodal information and fail to utilize their complementary information, resulting in limited segmentation accuracy. To address this problem, we design an Attention Bottlenecks Fusion Network (ABFNet) to segment brain tumor based on MRI by fusing multimodal complementary information. Firstly, based on the encoder-decoder architecture, we utilize multiple CNN encoders to extract modality-specific features. Then, a Bottleneck Fusion Transformer (BFT) module is introduced, which uses Transformer-style to fuse information from different modalities through a small number of potential bottleneck tokens, forcing each modality to condense the most necessary information and share it. Lastly, Fusion Connection Gating (FCG) is proposed to fuse modality-specific features from different levels through concatenation and convolutional operations. Multi-scale features utilized in FCG can enrich features and enhance the discriminability of every modality. Experiments are conducted on datasets BraTS2020 and BraTS2018. On BraTS2020, the DSC of our method on complete tumor, core tumor and enhancing tumor are 92.95%, 91.64% and 79.62%. On BraTS2018, the corresponding results are 90.12%, 83.40% and 70.56%. And experiments of modalities missing situations are conducted, showing the robustness and effectiveness of our method.