MRFD：基于自一致性的多区域融合解码以缓解LVLMs中的幻觉
MRFD: Multi-Region Fusion Decoding with Self-Consistency for Mitigating Hallucinations in LVLMs
AI总结：
提出无需训练的MRFD解码方法，通过交叉注意力识别图像显著区域并基于JSD计算可靠性权重，进行区域预测的一致性感知融合，有效缓解LVLMs的幻觉问题并提升响应事实性。
摘要：
大型视觉语言模型（LVLMs）在多模态任务中表现出色。然而，由于验证图像不同区域信息的能力有限，它们经常产生幻觉——即与视觉输入不一致的文本。为了解决这一问题，我们提出了多区域融合解码（MRFD），这是一种无需训练的解码方法，通过建模区域间一致性来改善事实基础。MRFD利用交叉注意力识别显著区域，为每个区域生成初始响应，并基于响应间的Jensen-Shannon散度（JSD）计算可靠性权重。这些权重引导使用受思维链推理启发的区域感知提示，对每个区域的预测进行一致性感知融合。在多个LVLMs和基准上的实验表明，MRFD显著减少了幻觉并提高了响应的事实性，且无需模型更新。
英文摘要：
Large Vision-Language Models (LVLMs) have shown strong performance across multimodal tasks. However, they often produce hallucinations -- text that is inconsistent with visual input, due to the limited ability to verify information in different regions of the image. To address this, we propose Multi-Region Fusion Decoding (MRFD), a training-free decoding method that improves factual grounding by modeling inter-region consistency. MRFD identifies salient regions using cross-attention, generates initial responses for each, and computes reliability weights based on Jensen-Shannon Divergence (JSD) among the responses. These weights guide a consistency-aware fusion of per-region predictions, using region-aware prompts inspired by Chain-of-Thought reasoning. Experiments across multiple LVLMs and benchmarks show that MRFD significantly reduces hallucinations and improves response factuality without requiring model updates.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
