A Time Series Transformer based method for the rotating machinery fault diagnosis

Fault diagnosis of rotating machinery is a significant engineering problem. In recent years, fault diagnosis methods have matured based on the Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN). However, these traditional models have the problem of Long-Term Dependencies, leading to their feature extraction ability defect. To address these issues, we proposed a new method based on the Time Series Transformer (TST) to recognize the fault modes of the various rotating machinery. In this paper, firstly, we design a new tokens sequences generation method that can handle data in 1D format, namely time series tokenizer. Then the TST combining time series tokenizer and Transformer is presented. The test results on the given datasets show that the proposed method has better fault identification capability than traditional CNN and RNN models. Secondly, the effect of structural hyperparameters on fault diagnosis performance, computational complexity, and parameters number of the TST is analyzed in detail through experiments. The influence laws of some hyperparameters are obtained as well. Finally, the feature vectors in the embedding space are visualized via the t-Distributed Stochastic Neighbor Embedding (t-SNE) dimensionality reduction method. On this basis, the working pattern of TST is explained to a certain extent. Moreover, we find that the feature vectors extracted by the proposed method show the best intra-class compactness and inter-class separability compared with CNN and RNN models by analyzing their distribution form, which further demonstrates the effectiveness of the proposed method.