arXiv 2512.075642025-12-08cs.CVcs.AIcs.CLcs.LG

迈向更可靠的人工智能：减少视觉-语言模型中的幻觉

Toward More Reliable Artificial Intelligence: Reducing Hallucinations in Vision-Language Models

Department of CS AI and Data Science（计算机科学与数据科学系）；ESEO Engineering School（ESEO工程学院）；Faculty of Law, Economy, Management（法学院、经济与管理学院）

Kassoum Sanogo, Renzo Ardiccioni

备注：: 24 pages, 3 figures, 2 tables. Training-free self-correction framework for vision-language models. Code and implementation details will be released at: https://github.com/kassoumsanogo1/self-correcting-vlm-re-Attention.git
发表信息：: The 4th National and International Academic Conference Celebrating the 20th Anniversary of Rajapruk University (2026)

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出无需训练的自我纠正框架，通过不确定性引导的视觉再注意力减少视觉-语言模型中的幻觉，提升响应准确性。

摘要：

视觉-语言模型（VLMs）经常生成看似合理但错误的图像内容声明。我们提出了一种无需训练的自我纠正框架，使VLMs能够通过不确定性引导的视觉再注意力逐步完善响应。我们的方法结合了多维不确定性量化（令牌熵、注意力分散、语义一致性、声明置信度）与注意力引导的未探索区域裁剪。该框架完全使用冻结的预训练VLMs运行，无需梯度更新。我们在POPE和MMHAL BENCH基准上使用Qwen2.5-VL-7B[23]架构验证了我们的方法。实验结果表明，与基线相比，我们的方法将幻觉率降低了9.8个百分点，同时在对抗性分割上将物体存在准确性提高了4.7个百分点。此外，定性分析证实，不确定性引导的再注意力在标准解码失败时能够通过视觉证据进行纠正。我们验证了该方法在Qwen2.5-VL-7B[23]上的有效性，并计划在未来版本中扩展到多样化的架构。我们发布代码和方法以促进未来可信多模态系统的研究。

英文摘要：

Vision-language models (VLMs) frequently generate hallucinated content plausible but incorrect claims about image content. We propose a training-free self-correction framework enabling VLMs to iteratively refine responses through uncertainty-guided visual re-attention. Our method combines multidimensional uncertainty quantification (token entropy, attention dispersion, semantic consistency, claim confidence) with attention-guided cropping of under-explored regions. Operating entirely with frozen, pretrained VLMs, our framework requires no gradient updates. We validate our approach on the POPE and MMHAL BENCH benchmarks using the Qwen2.5-VL-7B [23] architecture. Experimental results demonstrate that our method reduces hallucination rates by 9.8 percentage points compared to the baseline, while improving object existence accuracy by 4.7 points on adversarial splits. Furthermore, qualitative analysis confirms that uncertainty-guided re-attention successfully grounds corrections in visual evidence where standard decoding fails. We validate our approach on Qwen2.5-VL-7B [23], with plans to extend validation across diverse architectures in future versions. We release our code and methodology to facilitate future research in trustworthy multimodal systems.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。