arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.14992cs.AIcs.MM

揭示短视频中跨模态一致性以检测虚假新闻

Exposing Cross-Modal Consistency for Fake News Detection in Short-Form Videos

Mohamed bin Zayed University of Artificial Intelligence(莫扎德·本·泽亚德人工智能大学);Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所)

Chong Tian, Yu Wang, Chenxu Yang, Junyi Guan, Zheng Lin, Yuhan Liu, Xiuying Chen, Qirong Ho

AI总结:

本文提出MAGIC3模型,通过多粒度跨模态一致性建模提升虚假新闻检测性能,实现高准确率与低成本的平衡。

摘要:

短视频平台是新闻传播的主要渠道,但也成为多模态虚假信息的温床。在FakeSV(中文)和FakeTT(英文)基准数据集上,研究发现真实视频在文本-视觉一致性上较高但文本-音频一致性较中等,而虚假视频则相反。此外,单一全局一致性评分可作为可解释轴,影响虚假概率和预测误差。基于这些观察,本文提出MAGIC3(模态对抗门控交互和一致性中心分类器),通过显式成对和全局一致性建模,结合跨模态注意力得到的token和帧级一致性信号,结合多风格LLM重写以获得风格鲁棒的文本表示,并采用不确定性感知分类器进行选择性VLM路由。使用预提取特征,MAGIC3在FakeSV和FakeTT上均优于最强的非VLM基线。虽然匹配VLM级别精度,两阶段系统在吞吐量上提高18-27倍,VRAM节省93%,提供强的成本性能权衡。

英文摘要:

Short-form video platforms are major channels for news but also fertile ground for multimodal misinformation where each modality appears plausible alone yet cross-modal relationships are subtly inconsistent, like mismatched visuals and captions. On two benchmark datasets, FakeSV (Chinese) and FakeTT (English), we observe a clear asymmetry: real videos exhibit high text-visual but moderate text-audio consistency, while fake videos show the opposite pattern. Moreover, a single global consistency score forms an interpretable axis along which fake probability and prediction errors vary smoothly. Motivated by these observations, we present MAGIC3 (Modal-Adversarial Gated Interaction and Consistency-Centric Classifier), a detector that explicitly models and exposes cross-tri-modal consistency signals at multiple granularities. MAGIC3 combines explicit pairwise and global consistency modeling with token- and frame-level consistency signals derived from cross-modal attention, incorporates multi-style LLM rewrites to obtain style-robust text representations, and employs an uncertainty-aware classifier for selective VLM routing. Using pre-extracted features, MAGIC3 consistently outperforms the strongest non-VLM baselines on FakeSV and FakeTT. While matching VLM-level accuracy, the two-stage system achieves 18-27x higher throughput and 93% VRAM savings, offering a strong cost-performance tradeoff.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑