MMFakeBench：面向LVLMs的混合来源多模态虚假信息检测基准
MMFakeBench: A Mixed-Source Multimodal Misinformation Detection Benchmark for LVLMs
AI总结：
针对现实世界多来源伪造并存的问题，提出首个混合来源多模态虚假信息检测基准MMFakeBench，涵盖3个来源和12种伪造类型，并评估6种方法及15种LVLMs，同时提出MMD-Agent方法提升检测准确性与泛化能力。
摘要：
当前的多模态虚假信息检测（MMD）方法通常假设每个样本具有单一来源和单一类型的伪造，这不足以应对现实世界中多种伪造来源并存的情况。缺乏针对混合来源虚假信息的基准阻碍了该领域的进展。为解决这一问题，我们引入了MMFakeBench，这是首个针对混合来源MMD的综合性基准。MMFakeBench包含3个关键来源：文本真实性扭曲、视觉真实性扭曲和跨模态一致性扭曲，以及12个虚假信息伪造类型的子类别。我们进一步在零样本设置下，对6种主流检测方法和15种大型视觉语言模型（LVLMs）在MMFakeBench上进行了广泛评估。结果表明，当前方法在这种具有挑战性且现实可行的混合来源MMD设置下表现不佳。此外，我们提出了MMD-Agent，一种整合LVLM智能体推理、行动和工具使用能力的新方法，显著提高了准确性和泛化能力。我们相信这项研究将推动未来对更现实的混合来源多模态虚假信息的研究，并为虚假信息检测方法提供公平的评估。
英文摘要：
Current multimodal misinformation detection (MMD) methods often assume a single source and type of forgery for each sample, which is insufficient for real-world scenarios where multiple forgery sources coexist. The lack of a benchmark for mixed-source misinformation has hindered progress in this field. To address this, we introduce MMFakeBench, the first comprehensive benchmark for mixed-source MMD. MMFakeBench includes 3 critical sources: textual veracity distortion, visual veracity distortion, and cross-modal consistency distortion, along with 12 sub-categories of misinformation forgery types. We further conduct an extensive evaluation of 6 prevalent detection methods and 15 Large Vision-Language Models (LVLMs) on MMFakeBench under a zero-shot setting. The results indicate that current methods struggle under this challenging and realistic mixed-source MMD setting. Additionally, we propose MMD-Agent, a novel approach to integrate the reasoning, action, and tool-use capabilities of LVLM agents, significantly enhancing accuracy and generalization. We believe this study will catalyze future research into more realistic mixed-source multimodal misinformation and provide a fair evaluation of misinformation detection methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
