Shap-PreBiNT: A Sentiment Analysis Model Based on Optimized Transformer

Confront the problems of the semantic representation, classification accuracy and efficiency of text sentiment analysis methods need to be optimized, this paper proposes an optimized text sentiment classification model (Shap-PreBiNT) based on Transformer. In the text embedding stage, a Shap-Word model is devised to improve the ability of text semantic representation. The Shapley-value method is introduced to calculate the contribution weight of words in sentences, which fused with Word2Vec word vector. In the feature extraction stage, a bidirectional normalization layer is designed to regulate the feature distribution from multi-dimension. In the stage of network structure optimization, a pre-normalization structure is adopted to stabilize the gradient norm and accelerate the convergence rate. The experimental results demonstrate that the proposed model has a better performance by comparing with other related models. On the IMDB English dataset, the classification accuracy and F1-score reach 94.87% and 94.83%, which are 1.48% and 1.47% higher than Transformer. On the ChnSentiCorp Chinese dataset, our model achieves the highest accuracy and F1-Score, which are 91.82% and 91.66%, respectively, and increased by 2.43% and 2.51%.