Multimodal fake news detection through data augmentation-based contrastive learning

During the information exploding era, news can be created or edited purposely for promoting the spreading of social influence. However, unverified or fabricated news can also spread unscrupulously, leading to serious consequences, such as poor decisions or even health risk. Thus, in order to discriminate the fake news, several fake news detection approaches have been proposed and the majority of these methods suffer from low efficacy of detection, due to the lack of multimodal information and the small data size. Hence, we develop a novel machine learning based framework, i.e., BERT-based back-Translation Text and Entire-image multimodal model with Contrastive learning (TTEC). In this framework, the text of news is first back-translated encouraging the framework to learn some general characteristics regarding a particular topic. Secondly, both textual and visual features are fed into a BERT-based model in order to produce multimodal features. Thirdly, the contrastive learning is utilized to derive more reasonable multimodal representations through utilizing similar news published in the past. Eventually, to demonstrate the effectiveness of the proposed framework, extensive experiments are conducted and the results show our method outperforms the state of art methods by 3.1% on Mac. F1 scores.