A novel spatiotemporal 3D CNN framework with multi-task learning for efficient structural damage detection

In recent years, convolutional neural networks (CNNs) have demonstrated promising results in detecting structural damage. However, their architectures often overlook spatial and temporal effects simultaneously. This limitation can result in the loss of valuable information and an incapability to fully capture the complexity of the data, ultimately leading to reduced accuracy and suboptimal performance. This study proposes an intuitive three-dimensional CNN architecture that takes into account vibration history along with sensor spatial relations based on their relative positions. Furthermore, a multi-task learning (MTL) approach is suggested, which is a powerful approach for performing multiple tasks with a single network. The proposed 3D CNN method has been employed to detect single and double damage cases in an experimental steel frame through conventional classification alongside the transfer learning (TL). Moreover, MTL is used to detect single and double damage scenarios with a single unified network, which evaluates damage presence in separate tasks. The 3D CNN fulfilled state-of-the-art performance and 100% accuracy in detecting structural damage in almost all experiments. Additionally, the MTL model achieved promising results even in the presence of severe imbalanced classes of data. Furthermore, it was observed that the utilization of TL resulted in a notable reduction of computation time by 68% and the number of trainable parameters by 90% with the same level of accuracy in double-damage cases.