Triple Pseudo-Siamese network with hybrid attention mechanism for welding defect detection

Most of the existing methods neglect their complementary relation and only use welding pool images to detect welding defects. Therefore, a new triple pseudo-siamese network to improve identification performance using their complementarity is proposed in this paper. The network consists of three branches that extract the effective features of welding pool image, sound, current and voltage, respectively. Moreover, we employ a convolutional block attention module (CBAM) and propose cross-modal attention (CMA) to focus on the vital regions of the welding pool image. To train our model, the experimental setup was first designed. Then, it was deployed in an actual body production workshop of high-speed train to collect welding pool images, as well as sound and current and voltage data. Hence, a new dataset for detecting welding defects is presented. Experimental results on the dataset indicate that the proposed model can effectively detect different defects and actions. Furthermore, when the output of the shallow layer, the layer near the input layer, is fed into the CBAM and the output of the deep layer, the layer near the output layer, is fed into the CMA. In addition, they are highly complementary to each other.