驯服幻觉：通过反事实视频生成提升MLLMs的视频理解
Taming Hallucinations: Boosting MLLMs' Video Understanding via Counterfactual Video Generation
AI总结：
通过反事实视频生成和对比训练，提升MLLMs对视频的理解能力并减少幻觉。
摘要：
多模态大语言模型（MLLMs）在视频理解方面取得了显著进展。然而，它们存在一个关键漏洞：过度依赖语言先验，这可能导致视觉不 grounded 的幻觉，尤其是在处理反事实视频时，这些视频违背常识。这种局限性源于文本和视频之间固有的数据不平衡，由于收集和标注反事实数据的成本高昂，因此难以解决。为此，我们引入了DualityForge，一种新颖的反事实数据合成框架，该框架利用可控的、基于扩散的视频编辑技术，将真实视频转换为反事实场景。通过将结构化的上下文信息嵌入视频编辑和问答生成过程，该框架能够自动产生高质量的QA配对，以及原始编辑视频配对用于对比训练。基于此，我们构建了DualityVidQA，一个大规模的视频数据集，旨在减少MLLM的幻觉。此外，为了充分利用我们配对数据的对比性质，我们提出了Duality-Normalized Advantage Training（DNA-Train），一种两阶段SFT-RL训练制度，其中RL阶段应用配对的ℓ₁优势归一化，从而实现更稳定和高效的策略优化。在DualityVidQA-Test上的实验表明，我们的方法在反事实视频上显著减少了模型幻觉，相对于Qwen2.5-VL-7B基线，相对改进为24.0%。此外，我们的方法在幻觉和通用任务基准上均取得了显著成效，表明了强大的泛化能力。我们将开源我们的数据集和代码。
英文摘要：
Multimodal Large Language Models (MLLMs) have made remarkable progress in video understanding. However, they suffer from a critical vulnerability: an over-reliance on language priors, which can lead to visual ungrounded hallucinations, especially when processing counterfactual videos that defy common sense. This limitation, stemming from the intrinsic data imbalance between text and video, is challenging to address due to the substantial cost of collecting and annotating counterfactual data. To address this, we introduce DualityForge, a novel counterfactual data synthesis framework that employs controllable, diffusion-based video editing to transform real-world videos into counterfactual scenarios. By embedding structured contextual information into the video editing and QA generation processes, the framework automatically produces high-quality QA pairs together with original-edited video pairs for contrastive training. Based on this, we build DualityVidQA, a large-scale video dataset designed to reduce MLLM hallucinations. In addition, to fully exploit the contrastive nature of our paired data, we propose Duality-Normalized Advantage Training (DNA-Train), a two-stage SFT-RL training regime where the RL phase applies pair-wise $\ell_1$ advantage normalization, thereby enabling a more stable and efficient policy optimization. Experiments on DualityVidQA-Test demonstrate that our method substantially reduces model hallucinations on counterfactual videos, yielding a relative improvement of 24.0% over the Qwen2.5-VL-7B baseline. Moreover, our approach achieves significant gains across both hallucination and general-purpose benchmarks, indicating strong generalization capability. We will open-source our dataset and code.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
