用于抑制扩散模型中不当图像概念的生长抑制剂
Growth Inhibitors for Suppressing Inappropriate Image Concepts in Diffusion Models
AI总结：
针对扩散模型从训练数据习得NSFW与侵权风格等隐式不当概念的问题，提出注入图像空间生长抑制剂并用适配器预测抑制尺度，实现免微调、低副作用的概念擦除。
摘要：
尽管文本到图像扩散模型具有出色的图像生成能力，但它们会无意中从海量且未经筛选的训练数据中学习不当概念，从而带来各种伦理和商业风险。具体而言，模型生成的图像可能出现不安全工作（NSFW）内容和风格版权侵权。导致这些问题的提示通常不包含明确的不安全词语；相反，它们包含隐晦且具有联想性的词语，被称为隐式不安全提示。现有方法在文本引导下直接微调模型，以改变扩散模型的认知，从而擦除不当概念。这不仅需要针对特定概念进行微调，还可能导致灾难性遗忘。为解决这些问题，我们探索不当概念在图像空间中的表征，并通过注入生长抑制剂将其引导至更合适的表征；这些抑制剂是在扩散过程中根据识别出的与不当概念相关的特征定制的。此外，由于不当概念的程度和范围各不相同，我们训练了一个适配器，以在注入过程中推断相应的抑制尺度。我们的方法有效捕捉细微词语在图像层面的表现，无需微调即可直接、高效地擦除目标概念。通过大量实验，我们证明本方法取得了更优的擦除效果，对其他概念几乎没有影响，同时保持了图像质量和语义。
英文摘要：
Despite their remarkable image generation capabilities, text-to-image diffusion models inadvertently learn inappropriate concepts from vast and unfiltered training data, which leads to various ethical and business risks. Specifically, model-generated images may exhibit not safe for work (NSFW) content and style copyright infringements. The prompts that result in these problems often do not include explicit unsafe words; instead, they contain obscure and associative terms, which are referred to as implicit unsafe prompts. Existing approaches directly fine-tune models under textual guidance to alter the cognition of the diffusion model, thereby erasing inappropriate concepts. This not only requires concept-specific fine-tuning but may also incur catastrophic forgetting. To address these issues, we explore the representation of inappropriate concepts in the image space and guide them towards more suitable ones by injecting growth inhibitors, which are tailored based on the identified features related to inappropriate concepts during the diffusion process. Additionally, due to the varying degrees and scopes of inappropriate concepts, we train an adapter to infer the corresponding suppression scale during the injection process. Our method effectively captures the manifestation of subtle words at the image level, enabling direct and efficient erasure of target concepts without the need for fine-tuning. Through extensive experimentation, we demonstrate that our approach achieves superior erasure results with little effect on other concepts while preserving image quality and semantics.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
