语言先验在衡量视觉-语言模型组合泛化能力中的作用
The Role of Linguistic Priors in Measuring Compositional Generalization of Vision-Language Models
AI总结：
本文研究视觉-语言模型的组合泛化评估问题，指出现有改进主要依赖语言先验而非图像信息，并提出一种去除语言先验的新型组合性度量。
摘要：
组合性是自然语言和图像等多种模态的共同属性，但多模态模型的组合泛化能力仍未被充分理解。本文识别出视觉-语言组合性的两个来源：语言先验以及图像与文本之间的相互作用。研究表明，当前提升组合泛化的尝试依赖于语言先验，而非图像中的信息。本文还提出了一种不依赖此类语言先验的新型组合性度量指标。
英文摘要：
Compositionality is a common property in many modalities including natural languages and images, but the compositional generalization of multi-modal models is not well-understood. In this paper, we identify two sources of visual-linguistic compositionality: linguistic priors and the interplay between images and texts. We show that current attempts to improve compositional generalization rely on linguistic priors rather than on information in the image. We also propose a new metric for compositionality without such linguistic priors.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
