Dr.V：一种通过细粒度时空定位诊断视频幻觉的层次化感知-时间-认知框架
Dr.V: A Hierarchical Perception-Temporal-Cognition Framework to Diagnose Video Hallucination by Fine-grained Spatial-Temporal Grounding
AI总结：
提出层次化框架Dr.V，通过Dr.V-Bench基准和Dr.V-Agent智能体，利用细粒度时空定位与认知推理诊断大型视频模型的幻觉，提升可解释性与可靠性。
摘要：
大型视频模型（LVMs）的最新进展显著提升了视频理解能力。然而，这些模型仍然存在幻觉问题，生成与输入视频冲突的内容。为解决这一问题，我们提出了Dr.V，一个涵盖感知、时间和认知层次的层次化框架，通过细粒度时空定位来诊断视频幻觉。Dr.V包含两个关键组件：基准数据集Dr.V-Bench和卫星视频智能体Dr.V-Agent。Dr.V-Bench包含来自4,974个视频的10k个实例，涵盖多种任务，每个实例均配有详细的时空标注。Dr.V-Agent通过在感知和时间层次系统应用细粒度时空定位，随后进行认知层次推理，来检测LVMs中的幻觉。这一逐步流程模拟了类人的视频理解，有效识别幻觉。大量实验表明，Dr.V-Agent在诊断幻觉的同时提升了可解释性和可靠性，为真实场景中鲁棒的视频理解提供了实用蓝图。所有数据和代码可在https://github.com/Eurekaleo/Dr.V获取。
英文摘要：
Recent advancements in large video models (LVMs) have significantly enhance video understanding. However, these models continue to suffer from hallucinations, producing content that conflicts with input videos. To address this issue, we propose Dr.V, a hierarchical framework covering perceptive, temporal, and cognitive levels to diagnose video hallucination by fine-grained spatial-temporal grounding. Dr.V comprises of two key components: a benchmark dataset Dr.V-Bench and a satellite video agent Dr.V-Agent. Dr.V-Bench includes 10k instances drawn from 4,974 videos spanning diverse tasks, each enriched with detailed spatial-temporal annotation. Dr.V-Agent detects hallucinations in LVMs by systematically applying fine-grained spatial-temporal grounding at the perceptive and temporal levels, followed by cognitive level reasoning. This step-by-step pipeline mirrors human-like video comprehension and effectively identifies hallucinations. Extensive experiments demonstrate that Dr.V-Agent is effective in diagnosing hallucination while enhancing interpretability and reliability, offering a practical blueprint for robust video understanding in real-world scenarios. All our data and code are available at https://github.com/Eurekaleo/Dr.V.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
