GDALR: Global Dual Attention and Local Representations in transformer for surface defect detection

Automated surface detection has gradually emerged as a promising and crucial inspection method in the industrial sector, greatly enhancing production quality and efficiency. However, current semantic network models based on Vision Transformers are primarily trained on natural images, which exhibit complex object textures and backgrounds. Additionally, pure Vision Transformers lack the ability to capture local representations, making it challenging to directly apply existing semantic segmentation models to industrial production scenarios. In this paper, we propose a novel transformer segmentation model specifically designed for surface defect detection in industrial settings. Firstly, we employ a Dual-Attention Transformer (DAT) as the backbone of our model. This backbone replaces the generic 2D convolution block with a new self-attention block in the Spatial Reduction Attention module (SRA), enabling the establishment of a global view for each layer. Secondly, we enhance the collection of local information during decoding by initializing the relative position between query and key pixels. Finally, to strengthen the salient defect structure, we utilize Pixel Shuffle to rearrange the Ground Truth (GT) in order to guide the feature maps at each scale. Extensive experiments are conducted on three publicly industrial datasets, and evaluation results describe the outstanding performance of our network in surface defect detection.