Multimodal Fusion Network with Contrary Latent Topic Memory for Rumor Detection

Rumors can mislead readers and even have a negative impact on public events, especially multimodal rumors with text and images, which attract readers’ attention more easily. Most existing methods focus on capturing specific characteristics of rumor events and have difficulty in identifying unknown rumor events. In this article, we propose a multimodal rumor-detection network (MRDN) for social rumor detection. MRDN combines the complementary information of text and images through the mechanism of multihead self-attention fusion, which allocates weight to different modalities to carry out feature fusion from multiple perspectives. Furthermore, MRDN exploits a contrary latent topic memory network to store semantic information about true and false patterns of rumors, which is useful for identifying upcoming new rumors. Extensive experiments conducted on three public datasets show that our multimodal rumor-detection method outperforms the state-of-the-art approaches.