从局部概念到普遍概念：评估视觉-语言模型的多元文化理解能力
From Local Concepts to Universals: Evaluating the Multicultural Understanding of Vision-Language Models
AI总结：
本文提出 GlobalRG 基准，通过跨普遍概念检索和文化视觉定位两项任务评估视觉-语言模型的多元文化理解能力，实验表明模型在不同文化间表现差异显著，凸显了提升多元文化理解的必要性。
摘要：
尽管视觉-语言模型近期取得了进展，但由于训练数据集中非西方文化图像的代表性不足，这些模型在非西方文化图像上的表现仍然不够理想。已有多个基准被提出用于测试模型的文化包容性，但它们的文化覆盖范围有限，且未能充分评估模型在普遍概念以及特定文化的局部概念上的文化多样性表现。为解决这些局限性，我们提出了 GlobalRG 基准，包含两项具有挑战性的任务：跨普遍概念检索和文化视觉定位。前一项任务需要从 50 个国家中检索普遍概念对应的文化多样化图像，后一项任务旨在从 15 个国家的图像中定位特定文化概念。我们对多种模型的评估表明，模型在不同文化之间的表现差异显著——这凸显了增强视觉-语言模型多元文化理解能力的必要性。
英文摘要：
Despite recent advancements in vision-language models, their performance remains suboptimal on images from non-western cultures due to underrepresentation in training datasets. Various benchmarks have been proposed to test models' cultural inclusivity, but they have limited coverage of cultures and do not adequately assess cultural diversity across universal as well as culture-specific local concepts. To address these limitations, we introduce the GlobalRG benchmark, comprising two challenging tasks: retrieval across universals and cultural visual grounding. The former task entails retrieving culturally diverse images for universal concepts from 50 countries, while the latter aims at grounding culture-specific concepts within images from 15 countries. Our evaluation across a wide range of models reveals that the performance varies significantly across cultures -- underscoring the necessity for enhancing multicultural understanding in vision-language models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
