arXiv 2508.031772026-07-07cs.CV

SAVER：通过风格感知视觉早期修正减轻大型视觉语言模型中的幻觉

SAVER: Mitigating Hallucinations in Large Vision-Language Models via Style-Aware Visual Early Revision

ROSE Lab, Interdisciplinary Graduate Programme, Nanyang Technological University, Singapore（南洋理工大学罗思实验室，跨学科研究生项目，新加坡）；ROSE Lab, School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore（南洋理工大学罗思实验室，电子与电气工程学院，新加坡）；City University of Hong Kong, Hong Kong SAR（香港城市大学，香港特别行政区）；Shanghai Jiao Tong University, China（上海交通大学，中国）；Singapore University of Technology and Design, Singapore（新加坡科技设计大学，新加坡）；VinUniversity, Hanoi, Vietnam（越南文大学，河内，越南）

Zhaoxu Li, Chenqi Kong, Yi Yu, Qiangqiang Wu, Xinghao Jiang, Ngai-Man Cheung, Bihan Wen, Alex Kot, Xudong Jiang

备注：: Accepted at AAAI 2026. 24 pages, 10 figures. Code: https://github.com/llizhaoxu/SAVER

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

研究大型视觉语言模型幻觉问题，构建含照片及风格化图像数据集并对比，提出SAVER机制，利用早期层反馈基于视觉注意力模式动态调整输出，减轻风格化图像引起的幻觉，实验证明其性能先进。

摘要：

大型视觉语言模型（LVLMs）在理解复杂视觉文本上下文方面取得重大突破，但幻觉问题限制其实际应用。以往方法忽视风格化图像风险。本文构建数据集，对比13个LVLMs，发现风格化图像更易引发幻觉。为此提出SAVER机制，实验表明其在减轻幻觉方面性能领先。

英文摘要：

Large Vision-Language Models (LVLMs) recently achieve significant breakthroughs in understanding complex visual-textual contexts. However, hallucination issues still limit their real-world applicability. Although previous mitigation methods effectively reduce hallucinations in photographic images, they largely overlook the potential risks posed by stylized images, which play crucial roles in critical scenarios such as game scene understanding, art education, and medical analysis. In this work, we first construct a dataset comprising photographic images and their corresponding stylized versions with carefully annotated caption labels. We then conduct head-to-head comparisons on both discriminative and generative tasks by benchmarking 13 advanced LVLMs on the collected datasets. Our findings reveal that stylized images tend to induce significantly more hallucinations than their photographic counterparts. To address this issue, we propose Style-Aware Visual Early Revision SAVER, a novel mechanism that dynamically adjusts LVLMs' final outputs based on the token-level visual attention patterns, leveraging early-layer feedback to mitigate hallucinations caused by stylized images. Extensive experiments demonstrate that SAVER achieves state-of-the-art performance in hallucination mitigation across various models, datasets, and tasks.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。