揭示地面ID:外部线索如何塑造多模态绑定
Uncovering Grounding IDs: How External Cues Shape Multimodal Binding
AI总结:
本文提出地面ID概念,揭示外部线索通过增强多模态绑定的注意力机制,提升跨模态定位精度并减少幻觉。
摘要:
大型视觉-语言模型(LVLMs)在多模态基准测试中表现出色,但在结构推理和精确定位方面仍存在局限。近期研究表明,添加简单的视觉结构,如分区和注释,可以提高准确性,但这些增益背后的内部机制仍不明确。我们研究了这一现象,提出了
英文摘要:
Large vision-language models (LVLMs) show strong performance across multimodal benchmarks but remain limited in structured reasoning and precise grounding. Recent work has demonstrated that adding simple visual structures, such as partitions and annotations, improves accuracy, yet the internal mechanisms underlying these gains remain unclear. We investigate this phenomenon and propose the concept of Grounding IDs, latent identifiers induced by external cues that bind objects to their designated partitions across modalities. Through representation analysis, we find that these identifiers emerge as consistent within-partition alignment in embedding space and reduce the modality gap between image and text. Causal interventions further confirm that these identifiers mediate binding between objects and symbolic cues. We show that Grounding IDs strengthen attention between related components, which in turn improves cross-modal grounding and reduces hallucinations. Taken together, our results identify Grounding IDs as a key symbolic mechanism that explains how external cues enhance multimodal binding and offer both interpretability and practical improvements.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。