arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.12944cs.CVcs.AI
收藏

扭曲或伪造?视频大语言模型中幻觉的调查

Distorted or Fabricated? A Survey on Hallucination in Video LLMs

Department of Electrical and Computer Engineering, Northeastern University(东北大学电气与计算机工程系);Khoury College of Computer Science, Northeastern University(东北大学科赫里计算机科学学院)

Yiyang Huang, Yitian Zhang, Yizhou Wang, Mingyuan Zhang, Liang Shi, Huimin Zeng, Yun Fu

AI总结:

本文调查视频大语言模型中的幻觉问题,提出动态扭曲和内容伪造两大类,分析其成因并提出改进方向,如开发运动感知视觉编码器和整合反事实学习技术。

摘要:

尽管视频-语言模型取得显著进展,但幻觉仍是视频大语言模型(Vid-LLMs)的持续挑战,指输出看似合理却与输入视频内容矛盾。本文全面分析了Vid-LLMs中的幻觉,并引入系统分类法,将其分为动态扭曲和内容伪造两大类,每类包含两个子类和代表性案例。基于此分类,本文回顾了幻觉评估与缓解的最新进展,涵盖关键基准、指标和干预策略。进一步分析了动态扭曲和内容伪造的根本原因,通常源于时间表示能力有限和视觉基础不足。这些见解为未来工作提供了方向,包括开发运动感知视觉编码器和整合反事实学习技术。本文汇总了零散进展,促进对Vid-LLMs幻觉的系统理解,为构建稳健可靠的视频-语言系统奠定基础。相关工作的最新整理列表见https://github.com/hukcc/Awesome-Video-Hallucination。

英文摘要:

Despite significant progress in video-language modeling, hallucinations remain a persistent challenge in Video Large Language Models (Vid-LLMs), referring to outputs that appear plausible yet contradict the content of the input video. This survey presents a comprehensive analysis of hallucinations in Vid-LLMs and introduces a systematic taxonomy that categorizes them into two core types: dynamic distortion and content fabrication, each comprising two subtypes with representative cases. Building on this taxonomy, we review recent advances in the evaluation and mitigation of hallucinations, covering key benchmarks, metrics, and intervention strategies. We further analyze the root causes of dynamic distortion and content fabrication, which often result from limited capacity for temporal representation and insufficient visual grounding. These insights inform several promising directions for future work, including the development of motion-aware visual encoders and the integration of counterfactual learning techniques. This survey consolidates scattered progress to foster a systematic understanding of hallucinations in Vid-LLMs, laying the groundwork for building robust and reliable video-language systems. An up-to-date curated list of related works is maintained at https://github.com/hukcc/Awesome-Video-Hallucination .

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑