探索文本到图像生成中内容审核的边界
Exploring the Boundaries of Content Moderation in Text-to-Image Generation
AI总结:
本文审计五个T2I平台及模型,发现官方安全指南与模型实际行为存在差异并导致过度审查,呼吁提升内容审核透明度并开展包容性对话。
摘要:
本文分析了五个文本到图像(T2I)生成平台的社区安全指南,并审计了五个T2I模型,重点关注与可能导致社会污名化领域中人类表征相关的提示词。当前研究主要通过限制有害内容的生成来确保安全,而我们的研究提供了互补的视角。我们认为安全概念难以定义和操作化,这反映在官方发布的安全指南与T2I模型实际行为之间的差异上,有时会导致过度审查。我们的发现呼吁提高透明度,并就平台的内容审核实践开展包容性对话,同时铭记其全球文化和社会影响。
英文摘要:
This paper analyzes the community safety guidelines of five text-to-image (T2I) generation platforms and audits five T2I models, focusing on prompts related to the representation of humans in areas that might lead to societal stigma. While current research primarily focuses on ensuring safety by restricting the generation of harmful content, our study offers a complementary perspective. We argue that the concept of safety is difficult to define and operationalize, reflected in a discrepancy between the officially published safety guidelines and the actual behavior of the T2I models, and leading at times to over-censorship. Our findings call for more transparency and an inclusive dialogue about the platforms' content moderation practices, bearing in mind their global cultural and social impact.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。