A UAV-Based Aircraft Surface Defect Inspection System via External Constraints and Deep Learning

In the field of aircraft maintenance, regular inspection of fuselage surface during the aircraft life cycle is a vital task to ensure the aircraft quality and flight safety. Currently, the inspection task is generally carried out manually in an indoor hangar, which is with low efficiency and reliability. In this article, a novel system based on the unmanned aerial vehicle (UAV) is presented to achieve automated aircraft surface inspection efficiently. The hardware is established with a lightweight and low-cost flight platform, on which a sensor containing an inertial measurement unit (IMU) and a camera is equipped for UAV localization. A high-resolution camera is equipped to collect images of fuselage for defect detection. Our inspection framework is mainly composed of two modules: the UAV localization module and the defect detection module. The localization module is designed to estimate the relative pose between the UAV and the aircraft, providing the foundation for image positioning on the aircraft surface. The existing visual–inertial odometry (VIO) approach is adopted to implement the pose estimation. To reduce the large drifts caused by the VIO approach, a novel method is proposed to deploy precalibrated ArUco markers around the aircraft, which serve as external constraints for the VIO objective to realize joint optimization of the camera pose. In addition, an adaptive weighting method is proposed, which takes into consideration the recognition effect of markers to balance the external constraints. The defect detection module aims to detect defects on the fuselage surface from images captured by the high-resolution camera, which is implemented based on deep learning. To address the issue of detection on a few training samples, the transfer learning strategy is exploited to first pretrain the model on a public defect dataset and then fine-tune it on our collected aircraft defect dataset. After detecting the defects, the defective region is reflected on the fuselage surface through the UAV pose on the corresponding frame provided by the localization module, realizing the accurate defect localization. Experiments on both the simulation environment and real data demonstrate the superiority of our proposed external localization module and the effectiveness of the crack detection module.