Early event detection in a deep-learning driven quality prediction model for ultrasonic welding

A goal in ultrasonic welding (USW) process monitoring is to accurately predict quality outcomes based on monitored signals. However, in most cases, knowing only that the USW process has failed is insufficient. Modern process automation should assess signal information and intercede to rectify process problems. Identification of when a process signal deviates from an acceptable final quality outcome, i.e., the time at which an abnormal event starts, facilitates control action or root cause analysis to bring it back to compliance. A long short-term memory (LSTM) recurrent neural network is proposed to monitor USW and other time-series signals and identify this point. This deep neural network is trained to classify quality outcomes from continuous signals. The process monitoring signals and their sampling time are divided into finite segments as input to this network. The time segment at which the process signal first converges to the final quality class prediction is identified using cross-entropy of the classification probabilities. This procedure is demonstrated using USW quality monitoring algorithms and robot motion failure detection. The examples show an LSTM network not only provides high accuracy for USW quality prediction, but also that the time of classification convergence is consistent with variance observed in USW weld quality factors. Moreover, classification convergence time was shown to be associated to specific robot motion failures, useful as input to adaptive learning. This work realizes deep-learning driven quality prediction and early event detection for quality classification problems, and provides the information necessary for adaptive control algorithms.