通过分阶段自奖励缓解多模态幻觉
Mitigating Multimodal Hallucination via Phase-wise Self-reward
AI总结：
本文提出PSRD框架，通过分阶段自奖励信号在推理时动态抑制幻觉，有效降低LLaVA-1.5-7B的幻觉率50%，在多个基准上优于现有方法。
摘要：
大型视觉-语言模型（LVLMs）仍面临视觉幻觉问题，即生成响应与视觉输入不一致。现有方法或依赖大规模标注数据微调，导致计算开销大，或采用静态事后策略，忽视幻觉出现的动态性。为此，本文引入新的自奖励框架，可在推理时动态抑制幻觉，无需外部监督。实证表明，视觉幻觉呈现分阶段动态模式，在每个语义阶段初期达到峰值。基于此，本文提出PSRD（分阶段自奖励解码），通过分阶段自奖励信号指导在线幻觉纠正。为减少解码过程中重复自评估的成本，将LVLMs的幻觉指导信号蒸馏为轻量级奖励模型。奖励模型随后在解码过程中提供实时指导，实现精准的幻觉抑制。所提PSRD显著降低LLaVA-1.5-7B的幻觉率50%，并在五个幻觉评估基准上优于现有事后方法。进一步分析证实，PSRD有效抑制幻觉传播，并在性能与推理效率之间实现高度可控的平衡。
英文摘要：
Large Vision-Language Models (LVLMs) still struggle with vision hallucination, where generated responses are inconsistent with the visual input. Existing methods either rely on large-scale annotated data for fine-tuning, which incurs massive computational overhead, or employ static post-hoc strategies that overlook the dynamic nature of hallucination emergence. To address these, we introduce a new self-rewarding framework, enabling dynamic hallucination mitigation at inference time without external supervision. On the empirical side, we reveal that visual hallucination exhibits phase-wise dynamic patterns, peaking at the onset of each semantic phase. Drawing on these insights, we propose \textbf{PSRD} (\textbf{Phase-wise \textbf{S}elf-\textbf{R}eward \textbf{D}ecoding) for online hallucination correction guided by phase-wise self-reward signals. To reduce the cost of repeated self-evaluation during decoding, we distill the hallucination guidance signal from LVLMs into a lightweight reward model. The reward model subsequently provides on-the-fly guidance for targeted intervention during the decoding process, enabling precise hallucination suppression. The proposed PSRD significantly reduces the hallucination rate of LLaVA-1.5-7B by 50.0% and consistently outperforms existing post-hoc methods across five hallucination evaluation benchmarks for four LVLMs. Further analysis confirms that PSRD effectively mitigates hallucination propagation and achieves a highly controllable trade-off between strong performance and inference efficiency.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
