构图的艺术：用于组合视觉定位的注意力正则化训练
The ART of Composition: Attention-Regularized Training for Compositional Visual Grounding
AI总结：
本文提出CompART，通过分解标题为以对象为中心的短语并构建复合短语，提升多对象定位能力，验证了其在多种视觉语言模型上的有效性。
摘要：
视觉-语言模型（VLMs）在隐式和显式视觉定位及相关任务上表现优异，但其能力通常在简单单对象短语上测试。我们发现，对于复杂多对象参考，定位性能会下降。这些限制主要源于训练目标依赖图像-标题对齐，其中直接多对象参考罕见，可能的参考数量理论上很大（指数级增长于对象数量），且归因困难。为此，无需额外注释，我们提出组合注意力正则化训练（CompART），将标题分解为以对象为中心的短语，并通过连接词配对构建复合短语。然后引入一种组合损失，鼓励复合短语引发的注意力等于其组成部分短语注意力之和，促进多对象定位的平衡。我们在四种VLM架构上评估CompART，涵盖对比基和生成基模型，四个多对象定位基准和两个VQA基准。CompART在多种VLM架构和数据集上一致提升了单对象和多对象参考的定位能力，并进一步展示了增强的视觉理解，尽管未专门训练此任务。
英文摘要：
Vision-Language Models (VLMs) have achieved strong performance on implicit and explicit visual grounding and related tasks. However, such abilities are generally tested on simple, single-object phrases. We find that grounding performance degrades for complex, multi-object references. These limitations largely arise from training objectives that leverage image-caption alignment, where direct multi-object references are rare, the number of possible such references is theoretically large (exponential in the number of objects), and attribution is difficult. To address this, without requiring any additional annotations, we propose Compositional Attention-Regularized Training (CompART), which decomposes captions into object-centric phrases and constructs composite phrases by pairing them with conjunctions. We then introduce a composition loss that encourages the attention induced by a composite phrase to equal the sum of the attentions of its constituent phrases, promoting balanced multi-object localization. We evaluate CompART across four VLM architectures, spanning both contrastive-based and generative-based models, on four benchmarks for multi-object grounding and two VQA benchmarks for general visual understanding. CompART consistently improves grounding for both single- and multi-object references across diverse VLM architectures and datasets, and further demonstrates enhanced visual understanding, as evidenced by gains on VQA, despite not being explicitly trained for this task.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
