INFACT：用于视频大语言模型中诱导忠实性和事实性幻觉诊断的基准测试
INFACT: A Diagnostic Benchmark for Induced Faithfulness and Factuality Hallucinations in Video-LLMs
AI总结：
INFACT是一个用于评估视频大语言模型中诱导忠实性和事实性幻觉的诊断基准测试，通过四种模式评估模型可靠性，发现基础模式准确性并不等同于诱导模式下的可靠性，证据篡改和时间干预显著降低模型性能。
摘要：
尽管进展迅速，视频大语言模型（Video-LLMs）仍因幻觉而不可靠，这些幻觉是与视频证据（忠实性）或可验证的世界知识（事实性）矛盾的输出。现有基准测试对事实性幻觉的覆盖有限，且主要在干净环境中评估模型。我们介绍了INFACT，一个包含9,800个QA实例的诊断基准测试，涵盖真实和合成视频，具有细粒度的忠实性和事实性分类。INFACT在四种模式下评估模型：基础（干净）、视觉退化、证据篡改和时间干预（用于顺序敏感项目）。在诱导模式下的可靠性通过抗性率（RR）和时间敏感度分数（TSS）量化。在14个代表性Video-LLMs上的实验表明，较高的基础模式准确性并不一定转化为诱导模式中的更高可靠性，证据篡改会降低稳定性，时间干预导致最大的退化。值得注意的是，许多开源基线在事实性上的TSS接近于零，表明在顺序敏感问题上存在显著的时间惯性。
英文摘要：
Despite rapid progress, Video Large Language Models (Video-LLMs) remain unreliable due to hallucinations, which are outputs that contradict either video evidence (faithfulness) or verifiable world knowledge (factuality). Existing benchmarks provide limited coverage of factuality hallucinations and predominantly evaluate models only in clean settings. We introduce \textsc{INFACT}, a diagnostic benchmark comprising 9{,}800 QA instances with fine-grained taxonomies for faithfulness and factuality, spanning real and synthetic videos. \textsc{INFACT} evaluates models in four modes: Base (clean), Visual Degradation, Evidence Corruption, and Temporal Intervention for order-sensitive items. Reliability under induced modes is quantified using Resist Rate (RR) and Temporal Sensitivity Score (TSS). Experiments on 14 representative Video-LLMs reveal that higher Base-mode accuracy does not reliably translate to higher reliability in the induced modes, with evidence corruption reducing stability and temporal intervention yielding the largest degradation. Notably, many open-source baselines exhibit near-zero TSS on factuality, indicating pronounced temporal inertia on order-sensitive questions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
