通过细粒度 AI 反馈检测并缓解大型视觉语言模型中的幻觉
Detecting and Mitigating Hallucination in Large Vision Language Models via Fine-Grained AI Feedback
AI总结：
针对 LVLM 幻觉依赖昂贵标注且检测粒度粗的问题，本文利用句子级细粒度 AI 反馈训练检测器，并通过检测-重写与 HSA-DPO 融入幻觉严重程度以缓解幻觉。
摘要：
快速发展的大型视觉语言模型（LVLMs）已在一系列多模态任务上展现出显著能力，但仍面临幻觉现象，即生成文本与给定上下文不一致，这严重限制了 LVLMs 的使用。以往大多数工作在粗粒度层面检测和缓解幻觉，或需要昂贵的标注（例如由专有模型或人类专家标注）。为解决这些问题，我们提出通过细粒度 AI 反馈检测并缓解 LVLMs 中的幻觉。基本思路是：我们利用专有模型生成一个小规模的句子级幻觉标注数据集，据此训练一个幻觉检测模型，使其能够执行句子级幻觉检测，覆盖主要幻觉类型（即对象、属性和关系）。随后，我们提出一种“先检测后重写”的流程，以自动构建用于训练幻觉缓解模型的偏好数据集。此外，我们提出区分幻觉严重程度，并引入幻觉严重程度感知直接偏好优化（Hallucination Severity-Aware Direct Preference Optimization，HSA-DPO），通过将幻觉严重程度纳入偏好学习来缓解 LVLMs 中的幻觉。大量实验证明了我们方法的有效性。
英文摘要：
The rapidly developing Large Vision Language Models (LVLMs) have shown notable capabilities on a range of multi-modal tasks, but still face the hallucination phenomena where the generated texts do not align with the given contexts, significantly restricting the usages of LVLMs. Most previous work detects and mitigates hallucination at the coarse-grained level or requires expensive annotation (e.g., labeling by proprietary models or human experts). To address these issues, we propose detecting and mitigating hallucinations in LVLMs via fine-grained AI feedback. The basic idea is that we generate a small-size sentence-level hallucination annotation dataset by proprietary models, whereby we train a hallucination detection model which can perform sentence-level hallucination detection, covering primary hallucination types (i.e., object, attribute, and relationship). Then, we propose a detect-then-rewrite pipeline to automatically construct preference dataset for training hallucination mitigating model. Furthermore, we propose differentiating the severity of hallucinations, and introducing a Hallucination Severity-Aware Direct Preference Optimization (HSA-DPO) for mitigating hallucination in LVLMs by incorporating the severity of hallucinations into preference learning. Extensive experiments demonstrate the effectiveness of our method.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
