arXiv 2503.196222025-03-25cs.CV

探索大型多模态模型在视频理解中的幻觉：基准、分析与缓解

Exploring Hallucination of Large Multimodal Models in Video Understanding: Benchmark, Analysis and Mitigation

University of Chinese Academy of Sciences（中国科学院大学）；University of Cincinnati（辛辛那提大学）；National University of Singapore（新加坡国立大学）；Beijing Jiaotong University（北京交通大学）

Hongcheng Gao, Jiashu Qu, Jingyi Tang, Baolong Bi, Yue Liu, Hongyu Chen, Li Liang, Li Su, Qingming Huang

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对视频理解中LMMs的幻觉问题，提出基准HAVEN，分析7个影响因素，并设计视频思考模型结合SRFT和TDPO缓解幻觉，显著提升准确率并降低偏差。

摘要：

大型多模态模型（LMMs）的幻觉问题，即提供看似正确但实际上错误的响应，限制了其可靠性和适用性。本文旨在研究LMMs在视频模态中的幻觉问题，视频模态是动态的，相比图像和文本等静态模态更具挑战性。基于此动机，我们首先提出了一个全面的基准测试HAVEN，用于评估LMMs在视频理解任务中的幻觉。该基准基于三个维度构建，即幻觉原因、幻觉方面和问题格式，共产生6K个问题。然后，我们通过16个LMMs在提出的基准上的实验，定量研究了7个影响幻觉的因素，例如视频持续时间、模型大小和模型推理。此外，受最近思考模型如OpenAI o1的启发，我们提出了一种视频思考模型，通过监督推理微调（SRFT）和直接偏好优化（TDPO）来缓解LMMs的幻觉——其中SRFT增强推理能力，而TDPO减少思考过程中的幻觉。大量实验和分析证明了其有效性。值得注意的是，它在幻觉评估上的准确率提高了7.65%，并将偏差分数降低了4.5%。代码和数据公开在https://github.com/Hongcheng-Gao/HAVEN。

英文摘要：

The hallucination of large multimodal models (LMMs), providing responses that appear correct but are actually incorrect, limits their reliability and applicability. This paper aims to study the hallucination problem of LMMs in video modality, which is dynamic and more challenging compared to static modalities like images and text. From this motivation, we first present a comprehensive benchmark termed HAVEN for evaluating hallucinations of LMMs in video understanding tasks. It is built upon three dimensions, i.e., hallucination causes, hallucination aspects, and question formats, resulting in 6K questions. Then, we quantitatively study 7 influential factors on hallucinations, e.g., duration time of videos, model sizes, and model reasoning, via experiments of 16 LMMs on the presented benchmark. In addition, inspired by recent thinking models like OpenAI o1, we propose a video-thinking model to mitigate the hallucinations of LMMs via supervised reasoning fine-tuning (SRFT) and direct preference optimization (TDPO)-- where SRFT enhances reasoning capabilities while TDPO reduces hallucinations in the thinking process. Extensive experiments and analyses demonstrate the effectiveness. Remarkably, it improves the baseline by 7.65% in accuracy on hallucination evaluation and reduces the bias score by 4.5%. The code and data are public at https://github.com/Hongcheng-Gao/HAVEN.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。