Sequential concrete crack segmentation using deep fully convolutional neural networks and data fusion

Algorithms that interpret images to locate surface defects, such as cracks, play a key role in an automated inspection system. That is the reason the success of convolutional neural networks (CNNs) in image object detection persuaded researchers to apply deep CNNs for visual surface crack detection. Among various deep learning architectures, encoder decoder architectures with fully convolutional networks (FCNs) are powerful tools for automatically segmenting inspection images and detecting crack maps. In this study the U-Net architecture, as a particular FCN, is trained using the available concrete crack datasets. The trained network is then employed to detect crack maps in a sequence of images taken from a concrete beam-column specimen under a cyclic load test. To enhance performance of the crack segmentation, instead of treating each image in the sequence independently, the detection results of the next stages of the experiment are used to determine the crack map at the current stage. By leveraging the fact that cracks propagate sequentially, a data fusion technique is proposed that updates crack maps by considering the outcome of the next steps. To realize this method, reference points on images are utilized to estimate the deformation of the structural members. The deformation information is then used to project the previously detected crack maps onto the current image. This makes it possible to aggregate current and future detections and achieve higher accuracy. The framework laid out in this study provides tools to filter out false positives and recover missed detections.