The abnormal conditions of the crystallization process seriously affect the crystal quality and the smooth operation of the process. Compared to the continuous steady process, it is a big challenge to realize the fault detection and diagnosis (FDD) in a batch or semi-batch crystallization process which is unsteady and nonlinear. In this paper, a coupled method combining convolutional neural network (CNN) with dynamic time warping (DTW) is proposed for FDD in semi-batch crystallization process based on temperature and flow supersaturation control (TF-SSC). DTW solves the problem that the data is unsteady in a semi-batch process. Different fault data produced by introducing disturbances are calculated through DTW to obtain the similarity which is steady. Then, the similarity of different operating states is preprocessed and classified by the CNN. Compared to the traditional CNN, Resnet18 and Inception10, DTW-CNN method has an outstanding performance in FDD, especially under a small number of samples.