ChainMPQ: 交错文本图像推理链用于缓解关系幻觉
ChainMPQ: Interleaved Text-Image Reasoning Chains for Mitigating Relation Hallucinations
AI总结：
ChainMPQ通过交错文本和图像推理链，利用积累的记忆减少大型视觉语言模型的关系幻觉，提升关系推理能力。
摘要：
尽管大型视觉语言模型（LVLMs）在多模态任务中表现出色，但幻觉现象仍然阻碍了其可靠性。在三种幻觉类型中，包括对象、属性和关系的幻觉，关系幻觉占最大比例但受到最少关注。为了解决这个问题，我们提出了ChainMPQ（多视角问题引导的图像和文本交错链），这是一种无需训练的方法，通过利用积累的文本和视觉记忆来提高LVLMs的关系推理能力。ChainMPQ首先从问题中提取主体和对象关键词，以增强相应的图像区域。然后构建多视角问题，这些问题聚焦于关系的三个核心组成部分：主体、对象以及连接它们的关系。这些问题依次输入模型，早期步骤的文本和视觉记忆为后续步骤提供支持上下文，从而形成图像和文本交错的链式结构，引导逐步的关系推理。在多个LVLMs和基准测试中的实验表明，ChainMPQ显著减少了关系幻觉，而消融研究进一步验证了其三个核心模块的有效性。
英文摘要：
While Large Vision-Language Models (LVLMs) achieve strong performance in multimodal tasks, hallucinations continue to hinder their reliability. Among the three categories of hallucinations, which include object, attribute, and relation, relation hallucinations account for the largest proportion but have received the least attention. To address this issue, we propose ChainMPQ (Multi-Perspective Questions guided Interleaved Chain of Image and Text), a training-free method that improves relational inference in LVLMs by utilizing accumulated textual and visual memories. ChainMPQ first extracts subject and object keywords from the question to enhance the corresponding image regions. It then constructs multi-perspective questions that focus on the three core components of a relationship: the subject, the object, and the relation that links them. These questions are sequentially input to the model, with textual and visual memories from earlier steps providing supporting context for subsequent ones, thereby forming an interleaved chain of images and text that guides progressive relational reasoning. Experiments on multiple LVLMs and benchmarks show that ChainMPQ substantially reduces relation hallucinations, while ablation studies further validate the effectiveness of its three core modules.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
