图像完美的不完美:文本到图像模型演进阴影下的安全、偏见与真实性
Image-Perfect Imperfections: Safety, Bias, and Authenticity in the Shadow of Text-To-Image Model Evolution
AI总结:
本文以 Stable Diffusion 为例研究文本到图像模型演进中的安全、偏见与真实性问题,发现更新虽减少不安全图像但加剧性别偏见、负面刻板印象在非白人族群间持续或转移,并揭示旧版假图像检测器失效问题,通过微调可恢复至少 96.6% 准确率。
摘要:
文本到图像模型(如 Stable Diffusion,简称 SD)会经历迭代更新,以提升图像质量并解决安全等问题。图像质量的提升易于评估。然而,模型更新如何解决既有问题以及是否引发新问题,仍未得到充分探索。本研究从安全、偏见和真实性三个视角,对文本到图像模型的演进进行了初步调查。我们以 Stable Diffusion 为中心的研究结果表明,模型更新呈现出喜忧参半的局面。虽然更新逐步减少了不安全图像的生成,但偏见问题——尤其是性别偏见——反而加剧。我们还发现,随着 SD 的更新,负面刻板印象要么在同一非白人种族群体内部持续存在,要么转移至其他非白人种族群体,而这些特征与白人种族群体的关联极少。此外,我们的评估揭示了 SD 更新带来的一个新问题:最初针对早期 SD 版本训练的最先进假图像检测器,难以识别由更新版本生成的假图像。我们表明,在更新版本生成的假图像上对这些检测器进行微调,可在不同 SD 版本上实现至少 96.6% 的准确率,从而解决这一问题。我们的见解凸显了持续努力以缓解不断演进的文本到图像模型中的偏见与脆弱性的重要性。
英文摘要:
Text-to-image models, such as Stable Diffusion (SD), undergo iterative updates to improve image quality and address concerns such as safety. Improvements in image quality are straightforward to assess. However, how model updates resolve existing concerns and whether they raise new questions remain unexplored. This study takes an initial step in investigating the evolution of text-to-image models from the perspectives of safety, bias, and authenticity. Our findings, centered on Stable Diffusion, indicate that model updates paint a mixed picture. While updates progressively reduce the generation of unsafe images, the bias issue, particularly in gender, intensifies. We also find that negative stereotypes either persist within the same Non-White race group or shift towards other Non-White race groups through SD updates, yet with minimal association of these traits with the White race group. Additionally, our evaluation reveals a new concern stemming from SD updates: State-of-the-art fake image detectors, initially trained for earlier SD versions, struggle to identify fake images generated by updated versions. We show that fine-tuning these detectors on fake images generated by updated versions achieves at least 96.6\% accuracy across various SD versions, addressing this issue. Our insights highlight the importance of continued efforts to mitigate biases and vulnerabilities in evolving text-to-image models.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。