Fake News Detection Based on Multimodal Fusion in Coterminal Bi-Space

Abstract

Social media platforms are flooded with unverified information, much of which comes from multiple sources of heterogeneous data. Therefore, effective detection and prevention of fake news is crucial. Aiming at the problem that current multimodal fake news detection methods ignore inter-modal correlation and heterogeneity, this paper proposes a hierarchical self-attention mechanism network model based on shared and specific dual space (hereafter referred to as co-space) and fused Kronecker product. First, the model uses VGG-19 and BERT to acquire visual and textual features and then inputs them into the common-specific space for modal interaction to acquire shared and specific features. The shared space uses maximized mean difference for learning shared information between two modalities and reducing inter-modal differences; the specific space uses orthogonal constraints with the aim of capturing private information between different modalities and reducing feature redundancy. Finally, the resulting features are fused by introducing a hierarchical self-attention mechanism based on the Kronecker product to achieve news truthfulness classification.