跨越文化鸿沟：探索与解锁文本生成图像模型的文化视角
Navigating Cultural Chasms: Exploring and Unlocking the Cultural POV of Text-To-Image Models
AI总结：
本研究构建三层文化本体与提示模板，结合多维度评估方法和跨语言多模型的CulText2I数据集，探究文本生成图像模型的文化编码机制，为其跨文化应用奠定基础。
摘要：
文本生成图像（Text-To-Image, TTI）模型（如DALL-E和StableDiffusion）已展现出卓越的基于提示的图像生成能力。由于语言是文化的载体，多语言编码器可能对这类模型的文化能动性产生重大影响。本研究通过从三个层级维度刻画文化——文化维度、文化领域和文化概念——来探究TTI模型中内嵌的文化感知。基于该本体论，我们推导了用于解锁TTI模型中文化知识的提示模板，并提出一套全面的评估技术，包括使用CLIP空间的内在评估、借助视觉问答（Visual-Question-Answer, VQA）模型的外在评估以及人工评估，以衡量TTI生成图像的文化内容。为支撑本研究，我们构建了CulText2I数据集，该数据集源自6种不同的TTI模型，覆盖10种语言。我们的实验针对TTI模型中文化编码本质的“做什么、是什么、哪一个、如何做”四类研究问题提供了洞见，为这类模型的跨文化应用铺平了道路。
英文摘要：
Text-To-Image (TTI) models, such as DALL-E and StableDiffusion, have demonstrated remarkable prompt-based image generation capabilities. Multilingual encoders may have a substantial impact on the cultural agency of these models, as language is a conduit of culture. In this study, we explore the cultural perception embedded in TTI models by characterizing culture across three hierarchical tiers: cultural dimensions, cultural domains, and cultural concepts. Based on this ontology, we derive prompt templates to unlock the cultural knowledge in TTI models, and propose a comprehensive suite of evaluation techniques, including intrinsic evaluations using the CLIP space, extrinsic evaluations with a Visual-Question-Answer (VQA) model and human assessments, to evaluate the cultural content of TTI-generated images. To bolster our research, we introduce the CulText2I dataset, derived from six diverse TTI models and spanning ten languages. Our experiments provide insights regarding Do, What, Which and How research questions about the nature of cultural encoding in TTI models, paving the way for cross-cultural applications of these models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
