超越美学：文本到图像模型中的文化能力
Beyond Aesthetics: Cultural Competence in Text-to-Image Models
AI总结：
针对文本到图像模型忽视文化能力的问题，提出CUBE基准，从文化意识与多样性两个维度评估，并揭示现有模型的显著差距，以促进全球适应性发展。
摘要：
文本到图像（T2I）模型正越来越多地被全球不同的社区所采用，这些社区利用它们来创建其独特文化的视觉表征。当前的T2I基准主要关注生成图像的忠实度、美学和逼真度，而忽视了文化能力这一关键维度。在这项工作中，我们引入了一个框架，从两个关键维度评估T2I模型的文化能力：文化意识（cultural awareness）和文化多样性（cultural diversity），并提出了一种可扩展的方法，结合结构化知识库和大型语言模型，构建一个大规模的文化工件数据集，以支持这一评估。具体而言，我们应用该方法构建了CUBE（CUltural BEnchmark for Text-to-Image models，文本到图像模型的文化基准），这是首个评估T2I模型文化能力的基准。CUBE涵盖了与8个国家相关的文化工件，这些国家分布在不同地理文化区域，并涉及三个概念：美食、地标和艺术。CUBE包括：1）CUBE-1K，一组高质量提示，用于评估文化意识；2）CUBE-CSpace，一个更大的文化工件数据集，作为评估文化多样性的基础。我们还引入了文化多样性作为T2I评估的新组成部分，利用质量加权的Vendi分数。我们的评估揭示了现有模型在不同国家文化意识方面的显著差距，并为欠指定提示下T2I输出的文化多样性提供了宝贵的见解。我们的方法可扩展到其他文化区域和概念，并能促进T2I模型的发展，以更好地服务于全球人口。
英文摘要：
Text-to-Image (T2I) models are being increasingly adopted in diverse global communities where they create visual representations of their unique cultures. Current T2I benchmarks primarily focus on faithfulness, aesthetics, and realism of generated images, overlooking the critical dimension of cultural competence. In this work, we introduce a framework to evaluate cultural competence of T2I models along two crucial dimensions: cultural awareness and cultural diversity, and present a scalable approach using a combination of structured knowledge bases and large language models to build a large dataset of cultural artifacts to enable this evaluation. In particular, we apply this approach to build CUBE (CUltural BEnchmark for Text-to-Image models), a first-of-its-kind benchmark to evaluate cultural competence of T2I models. CUBE covers cultural artifacts associated with 8 countries across different geo-cultural regions and along 3 concepts: cuisine, landmarks, and art. CUBE consists of 1) CUBE-1K, a set of high-quality prompts that enable the evaluation of cultural awareness, and 2) CUBE-CSpace, a larger dataset of cultural artifacts that serves as grounding to evaluate cultural diversity. We also introduce cultural diversity as a novel T2I evaluation component, leveraging quality-weighted Vendi score. Our evaluations reveal significant gaps in the cultural awareness of existing models across countries and provide valuable insights into the cultural diversity of T2I outputs for under-specified prompts. Our methodology is extendable to other cultural regions and concepts, and can facilitate the development of T2I models that better cater to the global population.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
