SEASON: 通过自诊断对比解码缓解视频大语言模型中的时间幻觉
SEASON: Mitigating Temporal Hallucination in Video Large Language Models via Self-Diagnostic Contrastive Decoding
AI总结：
SEASON通过自诊断对比解码方法，无需训练即可提升视频大语言模型在时间信息处理上的准确性与可靠性。
摘要：
视频大语言模型（VideoLLMs）在视频理解方面取得了显著进展。然而，这些模型在响应用户查询时仍难以有效感知和利用视频中丰富的时序信息。因此，它们经常生成时间不一致或因果不合理的事件描述，导致严重的幻觉问题。尽管大多数先前研究集中在空间幻觉（例如对象不匹配），但视频理解中的时间推理仍相对研究较少。为了解决这个问题，我们提出了自诊断对比解码（SEASON），一种无需训练的方法，能够自适应地增强每个输出标记的时序和空间忠实性。它通过动态诊断每个标记的幻觉倾向，并对其对应的时间和空间负样本应用自适应对比解码来实现这一目标。广泛的实验表明，SEASON在三个幻觉检验基准上优于所有现有的无训练幻觉缓解方法，同时在四个通用视频理解基准上进一步提高了VideoLLMs的性能。代码将在接受后发布。
英文摘要：
Video Large Language Models (VideoLLMs) have shown remarkable progress in video understanding. However, these models still struggle to effectively perceive and exploit rich temporal information in videos when responding to user queries. Therefore, they often generate descriptions of events that are temporal inconsistent or causally implausible, causing severe hallucination issues. While most prior studies have focused on spatial hallucinations (e.g. object mismatches), temporal reasoning in video understanding remains relatively underexplored. To address this issue, we propose Self-Diagnostic Contrastive Decoding (SEASON), a training-free method that adaptively enhances temporal and spatial faithfulness for each output token. It achieves this by dynamically diagnosing each token's hallucination tendency and applying adaptive contrastive decoding against its corresponding temporal and spatial negatives. Extensive experiments demonstrate that SEASON outperforms all existing training-free hallucination mitigation approaches on three hallucination examination benchmarks, while further improves VideoLLMs across four general video understanding benchmarks. The code will be released upon acceptance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
