arXiv 2511.134202026-07-21cs.CV

VOPE：重新审视视觉语言模型在自愿想象任务中的幻觉现象

VOPE: Revisiting Hallucination of Vision-Language Models in Voluntary Imagination Task

Key Laboratory of AI Safety of CAS, Institute of Computing Technology, Chinese Academy of Sciences (CAS)（中国科学院人工智能安全重点实验室，计算技术研究所，中国科学院）；University of Chinese Academy of Sciences（中国科学院大学）；Zhongguancun Academy（中关村学院）

Xingming Long, Jie Zhang, Shiguang Shan, Xilin Chen

备注：: Accepted at ACM MM 2026 Dataset Track

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

研究聚焦视觉语言模型在自愿想象任务中的幻觉问题，引入VOPE评估基准，通过构建数据集应用于主流模型和缓解方法，发现多数模型幻觉严重，现有缓解方法效果有限，为该领域未来研究指明重要方向。

摘要：

大多数关于大型视觉语言模型（LVLMs）幻觉的研究集中在禁止图像中不存在输出的事实描述任务上。然而，尽管类似人类的认知能力对现实世界生成应用至关重要，但自愿想象任务（如故事写作）中的幻觉却很少受到关注。为解决这一局限，我们引入了自愿想象物体存在评估（VOPE）——一种基于重新检查的评估基准，用于评估LVLMs在自愿想象任务中的基础行为。具体而言，VOPE提出基于重新检查的问题，以评估LVLM如何在自身响应中解释想象物体的存在。VOPE基于模型对生成物体存在判断的正确性来识别幻觉，而非惩罚想象内容本身。基于此，我们构建了一个涵盖不同自愿想象水平的字幕、推理和写作任务的数据集。我们将VOPE应用于几种主流LVLMs和幻觉缓解方法，揭示了两个关键发现：（1）大多数LVLMs在自愿想象过程中大量产生幻觉，且在想象物体的存在评估中表现明显不佳；（2）现有的幻觉缓解方法在自愿想象任务中的效果有限，这使其成为未来研究的一个重要方向。

英文摘要：

Most research on hallucinations in Large Vision-Language Models (LVLMs) focuses on factual description tasks that prohibit any output absent from the image. However, little attention has been paid to hallucinations in voluntary imagination tasks, such as story writing, despite this human-like cognitive ability being essential for real-world generative applications. To address this limitation, we introduce Voluntary-imagined Object Presence Evaluation (VOPE) -- a recheck-based evaluation benchmark for assessing LVLMs' grounding behavior in voluntary imagination tasks. Specifically, VOPE poses recheck-based questions to evaluate how an LVLM interprets the presence of the imagined objects in its own response. Rather than penalizing the imagined content itself, VOPE identifies hallucinations based on the correctness of the model's presence judgments for the generated objects. Built on this idea, we construct a dataset covering captioning, reasoning, and writing tasks with different levels of voluntary imagination. We apply VOPE to several mainstream LVLMs and hallucination mitigation methods, revealing two key findings: (1) most LVLMs hallucinate heavily during voluntary imagination, and their performance in presence evaluation is notably poor on imagined objects; (2) existing hallucination mitigation methods show limited effect in voluntary imagination tasks, making this an important direction for future research.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。