ConceptPrune:通过技能神经元剪枝在扩散模型中进行概念编辑
ConceptPrune: Concept Editing in Diffusion Models via Skilled Neuron Pruning
AI总结:
本文提出免训练方法ConceptPrune,通过识别并剪枝扩散模型中负责生成不良概念的关键神经元区域实现概念遗忘,仅需剪枝约0.12%权重即可高效擦除多概念并抵御对抗性攻击。
摘要:
尽管大规模文本到图像扩散模型展示了令人印象深刻的图像生成能力,但对其潜在被滥用于生成不安全内容、侵犯版权以及延续社会偏见存在重大担忧。最近,文本到图像生成社区开始通过从预训练模型中编辑或遗忘不想要的概念来解决这些问题。然而,这些方法通常涉及数据密集且低效的微调,或利用各种形式的词元重映射,使其容易受到对抗性越狱攻击。在本文中,我们提出了一种简单有效的免训练方法 ConceptPrune,其中我们首先识别预训练模型中负责生成不想要概念的关键区域,从而通过权重剪枝促进直接的概念遗忘。在包括艺术风格、裸露、对象擦除和性别去偏等一系列概念上的实验表明,通过剪枝极小部分(约占总权重0.12%)的权重,可以高效擦除目标概念,实现多概念擦除,并对各种白盒和黑盒对抗性攻击具有鲁棒性。
英文摘要:
While large-scale text-to-image diffusion models have demonstrated impressive image-generation capabilities, there are significant concerns about their potential misuse for generating unsafe content, violating copyright, and perpetuating societal biases. Recently, the text-to-image generation community has begun addressing these concerns by editing or unlearning undesired concepts from pre-trained models. However, these methods often involve data-intensive and inefficient fine-tuning or utilize various forms of token remapping, rendering them susceptible to adversarial jailbreaks. In this paper, we present a simple and effective training-free approach, ConceptPrune, wherein we first identify critical regions within pre-trained models responsible for generating undesirable concepts, thereby facilitating straightforward concept unlearning via weight pruning. Experiments across a range of concepts including artistic styles, nudity, object erasure, and gender debiasing demonstrate that target concepts can be efficiently erased by pruning a tiny fraction, approximately 0.12% of total weights, enabling multi-concept erasure and robustness against various white-box and black-box adversarial attacks.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。