通过约束感知提示缓解多模态空间关系中的幻觉问题
Mitigating Hallucinations in Multimodal Spatial Relations through Constraint-Aware Prompting
AI总结：
针对大型视觉语言模型存在的空间关系幻觉问题，提出融入双向与传递性约束的约束感知提示框架，在三个常用数据集上性能优于现有方法，为约束缓解空间幻觉提供了更多可能。
摘要：
空间关系幻觉是大型视觉语言模型（LVLMs）长期存在的挑战，会导致模型对图像中物体的位置和空间构型生成错误预测。为解决这一问题，我们提出了一种约束感知提示框架，旨在减少空间关系幻觉。具体而言，我们引入两类约束：（1）双向约束，用于确保成对物体关系的一致性；（2）传递性约束，用于强化多个物体间的关系依赖。通过融入这些约束，LVLMs能够生成空间上更连贯、更一致的输出。我们在三个广泛使用的空间关系数据集上评估了所提方法，结果表明其性能优于现有方法。此外，我们对各类双向关系分析方案和传递性参考选择进行了系统分析，凸显了我们的方法在融入约束以缓解空间关系幻觉方面的更大潜力。
英文摘要：
Spatial relation hallucinations pose a persistent challenge in large vision-language models (LVLMs), leading to generate incorrect predictions about object positions and spatial configurations within an image. To address this issue, we propose a constraint-aware prompting framework designed to reduce spatial relation hallucinations. Specifically, we introduce two types of constraints: (1) bidirectional constraint, which ensures consistency in pairwise object relations, and (2) transitivity constraint, which enforces relational dependence across multiple objects. By incorporating these constraints, LVLMs can produce more spatially coherent and consistent outputs. We evaluate our method on three widely-used spatial relation datasets, demonstrating performance improvements over existing approaches. Additionally, a systematic analysis of various bidirectional relation analysis choices and transitivity reference selections highlights greater possibilities of our methods in incorporating constraints to mitigate spatial relation hallucinations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
