视觉-语言模型能理解复合名词吗？
Do Vision-Language Models Understand Compound Nouns?
AI总结：
本文构建Compun基准评估CLIP等视觉-语言模型对复合名词的理解能力，并提出用大语言模型生成多样化描述替代手写模板，将CLIP在Compun上的复合名词理解性能提升8.25%。
摘要：
诸如CLIP这类使用对比损失训练的开词汇视觉-语言模型（VLM），已成为文本到图像检索的一种有前景的新范式。然而，VLM能否像理解名词（例如lab）一样理解复合名词（CN）（例如lab coat）？我们构建了Compun，一个包含400个独特且常用复合名词的新基准，用于评估VLM在解释复合名词方面的有效性。Compun基准对VLM的文本到图像检索能力提出挑战：给定一个包含复合名词的文本提示，任务是在一对干扰图像中选出正确显示该复合名词的图像，而干扰图像分别显示构成该复合名词的各个名词。接下来，我们进行了深入分析，以揭示CLIP对某些类型复合名词理解的局限性。最后，我们提出了一个替代框架，超越了CLIP类模型广泛使用的手写文本提示模板。我们采用大语言模型生成多个多样化描述，这些描述将复合名词作为描述场景中的对象包含在内。我们提出的方法在Compun上将CLIP的复合名词理解能力提升了8.25%。代码和基准可在以下地址获取：https://github.com/sonalkum/Compun
英文摘要：
Open-vocabulary vision-language models (VLMs) like CLIP, trained using contrastive loss, have emerged as a promising new paradigm for text-to-image retrieval. However, do VLMs understand compound nouns (CNs) (e.g., lab coat) as well as they understand nouns (e.g., lab)? We curate Compun, a novel benchmark with 400 unique and commonly used CNs, to evaluate the effectiveness of VLMs in interpreting CNs. The Compun benchmark challenges a VLM for text-to-image retrieval where, given a text prompt with a CN, the task is to select the correct image that shows the CN among a pair of distractor images that show the constituent nouns that make up the CN. Next, we perform an in-depth analysis to highlight CLIPs' limited understanding of certain types of CNs. Finally, we present an alternative framework that moves beyond hand-written templates for text prompts widely used by CLIP-like models. We employ a Large Language Model to generate multiple diverse captions that include the CN as an object in the scene described by the caption. Our proposed method improves CN understanding of CLIP by 8.25% on Compun. Code and benchmark are available at: https://github.com/sonalkum/Compun
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
