R-CoV：区域感知的验证链用于减轻LVLMs中的物体幻觉
R-CoV: Region-Aware Chain-of-Verification for Alleviating Object Hallucinations in LVLMs
AI总结：
R-CoV通过区域感知的验证链方法，减轻LVLMs中的物体幻觉问题，采用六步流程，无需额外训练即可集成至多种LVLMs，实验表明有效缓解幻觉。
摘要：
大型视觉-语言模型（LVLMs）在多种多模态理解和推理任务中表现出色，但仍然面临物体幻觉问题，即在视觉输入中声称不存在的物体。为解决这一挑战，我们提出了区域感知的验证链（R-CoV），一种视觉验证链方法，用于在事后减轻LVLMs中的物体幻觉。受人类如何理解复杂视觉信息的启发，我们从LVLMs自身提取区域级处理，并将其用作链式提示以检测和减轻其自身的物体幻觉。具体而言，R-CoV包含六个步骤：初始响应生成、实体提取、坐标生成、区域描述、验证执行和最终响应生成。作为一种简单而有效的方法，R-CoV可以无缝集成到各种LVLMs中，无需额外训练，也不依赖外部检测模型。在多个广泛使用的幻觉基准测试中进行的大量实验表明，R-CoV可以显著减轻LVLMs中的物体幻觉。项目页面：https://github.com/Jiahao000/R-CoV。
英文摘要：
Large vision-language models (LVLMs) have demonstrated impressive performance in various multimodal understanding and reasoning tasks. However, they still struggle with object hallucinations, i.e., the claim of nonexistent objects in the visual input. To address this challenge, we propose Region-aware Chain-of-Verification (R-CoV), a visual chain-of-verification method to alleviate object hallucinations in LVLMs in a post-hoc manner. Motivated by how humans comprehend intricate visual information -- often focusing on specific image regions or details within a given sample -- we elicit such region-level processing from LVLMs themselves and use it as a chaining cue to detect and alleviate their own object hallucinations. Specifically, our R-CoV consists of six steps: initial response generation, entity extraction, coordinate generation, region description, verification execution, and final response generation. As a simple yet effective method, R-CoV can be seamlessly integrated into various LVLMs in a training-free manner and without relying on external detection models. Extensive experiments on several widely used hallucination benchmarks across multiple LVLMs demonstrate that R-CoV can significantly alleviate object hallucinations in LVLMs. Project page: https://github.com/Jiahao000/R-CoV.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
