arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 2401.16092cs.CLcs.CYcs.LG
收藏

多语言文本到图像生成放大了性别刻板印象,而提示工程可能无济于事

Multilingual Text-to-Image Generation Magnifies Gender Stereotypes and Prompt Engineering May Not Help You

TU Darmstadt(图宾根大学);Ontocord(Ontocord公司);Technical University of Munich (TUM)(慕尼黑技术大学);Munich Center for Machine Learning(慕尼黑机器学习中心);CERTAIN(CERTAIN公司);DFKI(德意志联邦防务研究院);Charles University Prague(布拉格查理大学);Centre for Cognitive Science, Darmstadt(达姆施塔特认知科学中心);Munich Data Science Institute(慕尼黑数据科学研究所)

Felix Friedrich, Katharina Hämmerl, Patrick Schramowski, Manuel Brack, Jindrich Libovicky, Kristian Kersting, Alexander Fraser

AI总结:

本文提出多语言性别偏见基准MAGBIG,发现多语言文本到图像模型不仅存在显著的性别刻板印象,且不同语言间表现差异显著,而提示工程策略缓解偏见效果有限并损害文本对齐。

摘要:

文本到图像生成模型近期在图像质量、灵活性和文本对齐方面取得了惊人的成果,因此被应用于数量快速增长的应用程序中。通过多语言能力的提升,更广泛的群体现在能够使用这项技术。然而,我们的结果表明,多语言模型与单语言模型一样,存在显著的性别偏见。此外,多语言生成在不同语言间会提供相似结果这一自然预期并不成立,语言之间存在重要差异。我们提出了一个新颖的基准 MAGBIG,旨在促进多语言模型中性别偏见的研究。我们使用 MAGBIG 调查多语言性对 T2I 模型中性别偏见的影响。为此,我们构建了多语言提示,要求提供具有特定职业或特征的人物肖像。我们的结果表明,模型不仅表现出强烈的性别偏见,而且在不同语言间表现不同。此外,我们调查了提示工程策略,如间接、中立的表述,以缓解这些偏见。不幸的是,这些方法效果有限,并导致文本到图像对齐变差。因此,我们呼吁对图像生成器中跨语言的多样化表征以及可控性进行更多研究,以解决有偏见的模型行为。

英文摘要:

Text-to-image generation models have recently achieved astonishing results in image quality, flexibility, and text alignment, and are consequently employed in a fast-growing number of applications. Through improvements in multilingual abilities, a larger community now has access to this technology. However, our results show that multilingual models suffer from significant gender biases just as monolingual models do. Furthermore, the natural expectation that multilingual models will provide similar results across languages does not hold up. Instead, there are important differences between languages. We propose a novel benchmark, MAGBIG, intended to foster research on gender bias in multilingual models. We use MAGBIG to investigate the effect of multilingualism on gender bias in T2I models. To this end, we construct multilingual prompts requesting portraits of people with a certain occupation or trait. Our results show that not only do models exhibit strong gender biases but they also behave differently across languages. Furthermore, we investigate prompt engineering strategies, such as indirect, neutral formulations, to mitigate these biases. Unfortunately, these approaches have limited success and result in worse text-to-image alignment. Consequently, we call for more research into diverse representations across languages in image generators, as well as into steerability to address biased model behavior.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑