全局上下文还是局部细节？面向幻觉缓解的自适应视觉 grounding
Global Context or Local Detail? Adaptive Visual Grounding for Hallucination Mitigation
AI总结：
本文提出PND框架，通过双路径对比在解码过程中增强视觉真实性，减少幻觉并提升描述细节，无需模型微调。
摘要：
视觉-语言模型（VLMs）常因过度依赖语言先验而产生物体幻觉，即生成与视觉现实矛盾的内容。我们引入正负解码（PND），一种无需训练的推理框架，直接干预解码过程以强制视觉一致性。PND基于VLMs中关键的注意力缺陷发现，即视觉特征经验上被低估。框架通过双路径对比：正路径利用多层注意力放大显著视觉证据，鼓励忠实描述；负路径识别并降级核心物体特征，创建强反事实，惩罚不 grounded 的生成。通过在每一步对比模型输出的两种视角，PND引导生成既语言上可能又视觉上真实的文本。在POPE、MME和CHAIR等基准上的广泛实验表明，PND在不需模型微调的情况下实现了最先进的性能，准确率提升达6.5%，显著减少物体幻觉并增强描述细节。该方法在包括LLaVA、InstructBLIP、InternVL和Qwen-VL在内的多种VLM架构上均表现良好。
英文摘要：
Vision-Language Models (VLMs) are frequently undermined by object hallucination--generating content that contradicts visual reality--due to an over-reliance on linguistic priors. We introduce Positive-and-Negative Decoding (PND), a training-free inference framework that intervenes directly in the decoding process to enforce visual fidelity. PND is motivated by our key finding of a critical attention deficit in VLMs, where visual features are empirically under-weighted. Our framework corrects this via a dual-path contrast: The positive path amplifies salient visual evidence using multi-layer attention to encourage faithful descriptions, directly counteracting the attention deficit. Simultaneously, the negative path identifies and degrades the core object's features to create a strong counterfactual, which penalizes ungrounded, prior-dominant generation. By contrasting the model's outputs from these two perspectives at each step, PND steers generation towards text that is not just linguistically probable, but visually factual. Extensive experiments on benchmarks like POPE, MME, and CHAIR show that PND achieves state-of-the-art performance with up to 6.5% accuracy improvement, substantially reducing object hallucination while also enhancing descriptive detail--all without requiring any model retraining. The method generalizes effectively across diverse VLM architectures including LLaVA, InstructBLIP, InternVL, and Qwen-VL.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
