arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2209.08891cs.CVcs.AIcs.CYcs.LG
收藏

利用文本到图像合成中的同形字进行文化偏见攻击

Exploiting Cultural Biases via Homoglyphs in Text-to-Image Synthesis

Technical University of Darmstadt(达姆施塔特工业大学);Hessian Center for AI (hessian.AI)(黑森人工智能中心);German Center for Artificial Intelligence (DFKI)(德国人工智能研究中心);Centre for Cognitive Science of Darmstadt(达姆施塔特认知科学中心)

Lukas Struppek, Dominik Hintersdorf, Felix Friedrich, Manuel Brack, Patrick Schramowski, Kristian Kersting

AI总结:

本文发现文本到图像模型会因输入非拉丁字符而产生文化偏见,并提出一种同形字遗忘方法微调文本编码器以防御此类攻击。

摘要:

文本到图像合成模型(如 DALL-E 2 和 Stable Diffusion)近期引起了学术界和公众的广泛关注。这些模型在以文本描述为条件时,能够生成描绘各种概念和风格的高质量图像。然而,这些模型从其海量训练数据中吸收了与特定 Unicode 脚本相关的文化特征,这并非显而易见。我们证明,只需在文本描述中插入单个非拉丁字符,常见模型就会在生成的图像中反映出文化刻板印象和偏见。我们对此行为进行了定性和定量分析,并确定模型的文本编码器是该现象的根本原因。此外,恶意用户或服务提供商可能会故意使图像生成产生偏见,通过用非拉丁脚本中外观相似的字符(即所谓的同形字)替换拉丁字符,从而制造种族主义刻板印象。为了缓解此类未被察觉的脚本攻击,我们提出了一种新颖的同形字遗忘方法来微调文本编码器,使其对同形字操作具有鲁棒性。

英文摘要:

Models for text-to-image synthesis, such as DALL-E~2 and Stable Diffusion, have recently drawn a lot of interest from academia and the general public. These models are capable of producing high-quality images that depict a variety of concepts and styles when conditioned on textual descriptions. However, these models adopt cultural characteristics associated with specific Unicode scripts from their vast amount of training data, which may not be immediately apparent. We show that by simply inserting single non-Latin characters in a textual description, common models reflect cultural stereotypes and biases in their generated images. We analyze this behavior both qualitatively and quantitatively, and identify a model's text encoder as the root cause of the phenomenon. Additionally, malicious users or service providers may try to intentionally bias the image generation to create racist stereotypes by replacing Latin characters with similarly-looking characters from non-Latin scripts, so-called homoglyphs. To mitigate such unnoticed script attacks, we propose a novel homoglyph unlearning method to fine-tune a text encoder, making it robust against homoglyph manipulations.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑