Transformer Based Prototype Learning for Weakly-Supervised Histopathology Tissue Semantic Segmentation

Weakly-supervised semantic segmentation for computational pathology has the great potential to alleviate the time-consuming and labor-intensive burden of manual pixel-level annotations. Existing methods relying on class activation map (CAM) to localize target objects suffer from two problems. First, most CAM-based models adopt convolutional neural networks, which cannot model the long-range dependencies of dispersed tissues. Second, CAM tends to focus on the most discriminative region of the object, resulting in incomplete segmentation results. In this paper, we propose a novel Transformer based weakly-supervised model for pixel-level tissue segmentation. The proposed model is able to capture global tissue feature relations by the self-attention mechanism in Transformer. For the issue of incomplete segmentation in CAM, we propose a patch-token prototype self-supervised learning approach to obtain more complete localization maps. Additionally, we introduce a self-refinement mechanism to dampen the falsely activated regions in the initial localization map. Extensive experiments on two histopathology datasets demonstrate that our proposed model achieves the state-of-the-art performance compared with other weakly-supervised methods.