SPCB-Net: A Multi-Scale Skin Cancer Image Identification Network Using Self-Interactive Attention Pyramid and Cross-Layer Bilinear-Trilinear Pooling

Deep convolutional neural networks have made some progress in skin lesion classification and cancer diagnosis, but there are still some problems to be solved, such as the challenge of small inter-class feature differences and large intra-class feature differences, which might limit the classification performance of the model as high-level and low-level features are not properly utilized. This paper proposes a multi-scale skin cancer image identification network using self-interactive attention pyramid and cross-layer bilinear-trilinear pooling(SPCB-Net), which mainly consists of three proposed sub-modules that are the self-interacting attention pyramid (SAP), the across-layer bilinear-trilinear pooling operation and the global average algorithm(GAA). The SPCB-Net is applied to two representative datasets of medical images in dermatology and histopathology (HAM10000 and NCT-CRC-HE-100K) to demonstrate the effectiveness of in the skin lesion classification. SPCB-Net(ResNet101) achieves 97.10% and 99.87% accuracy on HAM10000 and NCT-CRC-HE-100K respectively, which are both achieved performance improvements of 0.4% compared to the state-of-the-art models. In addition, a large number of experiments on HAM10000 show that the interactive attention pyramid(SPA) proposed in this paper is superior to the common attention module, and the method with a cross-layer bilinear-trilinear pooling is superior to the cross-layer trilinear pooling method. SPCB-Net is configured on Vgg19 and ResNet101 to evaluate the effectiveness of our proposed module. The experimental results show that SPCB-Net has shown state-of-the-art performance in the two field of dermatology and histopathology. Therefore, it is not only well qualified for the task of identifying skin cancer image but also has the potential to identify skin cancer by identifying pathological tissue.