A Semi-Supervised Learning Framework for TRIZ-Based Chinese Patent Classification

Automatic patent classification based on the TRIZ inventive principles is essential for patent management and industrial analysis. However, acquiring labels for deep learning methods is extraordinarily difficult and costly. This paper proposes a new two-stage semi-supervised learning framework called TRIZ-ESSL, which stands for Enhanced Semi-Supervised Learning for TRIZ. TRIZ-ESSL makes full use of both labeled and unlabeled data to improve the prediction performance. TRIZ-ESSL takes the advantages of semi-supervised sequence learning and mixed objective function, a combination of cross-entropy, entropy minimization, adversarial and virtual adversarial loss functions. Firstly, TRIZ-ESSL uses unlabeled data to train a recurrent language model. Secondly, TRIZ-ESSL initializes the weights of the LSTM-based model with the pre-trained recurrent language model and then trains the text classification model using mixed objective function on both labeled and unlabeled sets. On 3 TRIZ-based classification tasks, TRIZ-ESSL outperforms the current popular semi-supervised training methods and Bert in terms of accuracy score.