SoS：多语言文本到图像生成中的表面与语义分析
SoS: Analysis of Surface over Semantics in Multilingual Text-To-Image Generation
AI总结：
本文分析了多语言文本到图像生成中表面与语义的偏差问题，揭示了T2I模型在不同语言下生成刻板印象图像的倾向，并提出新的度量方法来量化这种现象。
摘要：
文本到图像（T2I）模型正被全球用户越来越多地使用。然而，先前的研究指出T2I模型对特定输入语言高度敏感——当面对非英语语言（即相同提示的不同表面形式）时，T2I模型往往生成具有文化刻板印象的图像，优先考虑表面而非提示的语义。然而，这种行为的全面分析，我们称之为表面过度语义（SoS），仍然缺失。我们提出了T2I模型SoS倾向的第一个分析。为此，我们创建了一组涵盖171种文化身份、翻译成14种语言的提示，并用其提示七个T2I模型。为了量化模型、语言和文化层面的SoS倾向，我们引入了一种新的度量方法，并分析了我们识别出的倾向如何在视觉上表现。我们证明除了一个模型外，所有模型在至少两种语言中都表现出强烈的表面倾向，这种效果在T2I文本编码器的各层中更加明显。此外，这些表面倾向经常与刻板印象的视觉表现相关联。
英文摘要：
Text-to-image (T2I) models are increasingly employed by users worldwide. However, prior research has pointed to the high sensitivity of T2I towards particular input languages - when faced with languages other than English (i.e., different surface forms of the same prompt), T2I models often produce culturally stereotypical depictions, prioritizing the surface over the prompt's semantics. Yet a comprehensive analysis of this behavior, which we dub Surface-over-Semantics (SoS), is missing. We present the first analysis of T2I models' SoS tendencies. To this end, we create a set of prompts covering 171 cultural identities, translated into 14 languages, and use it to prompt seven T2I models. To quantify SoS tendencies across models, languages, and cultures, we introduce a novel measure and analyze how the tendencies we identify manifest visually. We show that all but one model exhibit strong surface-level tendency in at least two languages, with this effect intensifying across the layers of T2I text encoders. Moreover, these surface tendencies frequently correlate with stereotypical visual depictions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
