Feature pyramid network with self-guided attention refinement module for crack segmentation

Automated pavement crack segmentation is challenging due to the random shape of cracks, complex background textures and the presence of miscellaneous objects. In this paper, we implemented a Self-Guided Attention Refinement module and incorporated it on top of a Feature Pyramid Network (FPN) to model long-range contextual information. The module uses multi-scale features integrated from different layers in the FPN to refine the features at each layer of the FPN using a self-attention mechanism. The module enables the earlier layers and deeper layers of FPN to suppress noise and increase the crack details, respectively. The proposed network obtains an F1 score of 79.43% and 96.19% on the Crack500 and CFD datasets, respectively. Furthermore, the network also generalizes better than other state-of-the-art methods when tested on uncropped Crack500 and field images using the weights trained on CFD. In addition, ablation tests using the Crack500 dataset are conducted. The Self-Guided Attention Refinement module increases the average F1 score and recall by 0.6% and 0.8% while roughly maintaining the average precision. From the ablation test, the inclusion of the Self-Guided Attention Refinement module allows the network to segment finer and more continuous cracks. Then, the module is incorporated on PANet, DeepLab v3+ and U-Net to verify the improvements made to FPN. The results show that the module improves the F1 score, precision and recall compared to the absence of it. Moreover, the Self-Guided Attention Refinement Module is compared with the Self-Adaptive Sparse Transform Module (SASTM). The results show that the Self-Guided Attention Refinement Module provides a more consistent improvement.