A real-time welding defect detection framework based on RT-DETR deep neural network

• A real-time welding defect detection framework based on the RT-DETR deep neural network is proposed. • By exploring three different data augmentation strategies, the most suitable method for weld defect detection is selected. • The proposed method outperforms other models in terms of both mAP and FPS. • The proposed method is validated in the WAAM weld defect detection. The quality of welds is critical to the safety and reliability of steel structure connections, underscoring the importance of accurate inspection during the welding process. To enhance inspection effectiveness, deep learning methods have gained popularity in weld defect detection for their ability to automatically learn and refine image features. However, the complex multi-stage training and inference process of these methods often fails to meet the requirements of real-time performance and accuracy. To address this problem, a framework based on the Real-Time DEtection TRansformer (RT-DETR) for deep learning-based welding defect detection is proposed. This framework improves the Transformer backbone by eliminating the most time-consuming non-maximum suppression (NMS) step, achieving real-time detection without sacrificing accuracy. A diverse welding dataset with 1,134 images from real-world manufacturing and construction environments was developed for model training and validation. In addition, three data enhancement algorithms were explored to enhance the model’s generalization ability. The model achieved detection accuracy scores of mAP@0.5 at 0.996 and mAP@0.5:0.95 at 0.801, with a detection speed of 67 frames per second (FPS). Compared to the previous Faster R-CNN, SSD, YOLOv5, YOLOv11 and DETR models, the proposed RT-DETR model demonstrates superior efficiency and accuracy. The proposed framework was further validated in the on-site inspections of metal additive manufacturing, and the results confirmed that the RT-DETR-based model meets the stringent requirements for real-time inspection in metal additive manufacturing.