Watch Closely: 通过解耦解码缓解大视觉-语言模型中的物体幻觉
Watch Closely: Mitigating Object Hallucinations in Large Vision-Language Models with Disentangled Decoding
AI总结：
通过解耦解码方法，无需训练即可缓解大视觉-语言模型中的物体幻觉问题，提升模型的视觉性能和准确性。
摘要：
大型视觉-语言模型（LVLMs）弥合了视觉和语言模态之间的差距，在各种领域展现出强大的潜力。然而，尽管取得了显著进展，LVLMs在物体识别任务中仍然存在严重的幻觉问题。这些模型往往无法准确识别某些物体，导致文本生成看起来流畅但与视觉内容不一致，这在实际应用中可能产生严重后果。最近，已有几种方法被提出以缓解LVLM的幻觉问题，但大多数仅关注减少语言模态中的幻觉。为了缓解语言和视觉模态中的幻觉，我们引入了幻觉解耦解码（HDD）方法，该方法无需训练。HDD通过分割图像并选择增强原始图像的图像来增强原始图像，同时利用空白图像来消除原始和分割图像中的语言先验幻觉。这种设计不仅减少了模型对语言先验的依赖，还增强了其视觉性能。(代码：https://github.com/rickeyhhh/Hallucination-Disentangled-Decoding)
英文摘要：
Large Vision-Language Models (LVLMs) bridge the gap between visual and linguistic modalities, demonstrating strong potential across a variety of domains. However, despite significant progress, LVLMs still suffer from severe hallucination issues in object recognition tasks. These models often fail to accurately identify certain objects, leading to text generation that appears fluent but does not correspond to the visual content, which can have serious consequences in real-world applications. Recently, several methods have been proposed to alleviate LVLM hallucinations, but most focus solely on reducing hallucinations in the language modality. To mitigate hallucinations in both the language and visual modalities, we introduce Hallucination Disentangled Decoding (HDD) method that requires no training. HDD enhances the original image by segmenting it and selecting images that augment the original, while also utilizing a blank image to eliminate language prior hallucinations in both the original and segmented images. This design not only reduces the model's dependence on language priors but also enhances its visual performance. (Code: https://github.com/rickeyhhh/Hallucination-Disentangled-Decoding)
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
