Weight Saliency search with Semantic Constraint for Neural Machine Translation attacks

Text adversarial attack is an effective way to improve the robustness of Neural Machine Translation (NMT) models. Existing NMT attack tasks are often completed by replacing words. However, most of previous works pursue a high attack success rate but produce semantic inconsistency sentences, leading to wrong translations even for humans. In this paper, we propose a Weight Saliency search with Semantic Constraint (WSSC) algorithm to make semantic consistency word modifications to the input sentence for black-box NMT attacks. Specifically, our WSSC has two major merits. First, it optimizes the word substitution with a word saliency method, which is helpful to reduce word replacement rate. Second, it constrains the objective function with a semantic similarity loss, ensuring every modification does not lead to significant semantic changes. We evaluate the effectiveness of the proposed WSSC by attacking three popular NMT models, i.e., T5, Marian, and BART, on three widely used datasets, i.e., WMT14, WMT16, and TED. Experimental results validate that our WSSC improves Attack Success Rate (ASR) by 4.02% and Semantic Similarity score (USE) by 1.28% on average. Besides, our WSSC also shows good properties in keeping grammar correctness and transfer attack.