arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2412.01244cs.CV
收藏

Concept Replacer:通过精确定位替换扩散模型中的敏感概念

Concept Replacer: Replacing Sensitive Concepts in Diffusion Models via Precision Localization

Fudan University(复旦大学);Purple Mountain Laboratories(紫金山实验室)

Lingyun Zhang, Yu Xie, Yanwei Fu, Ping Chen

AI总结:

本文提出 Concept Replacer,通过少样本概念定位器与无需训练的 DPCA 模块,在扩散模型去噪过程中精确替换敏感概念并保持非目标区域稳定。

摘要:

随着大规模扩散模型不断发展,它们擅长生成高质量图像,但往往会产生不良内容,例如色情露骨或暴力内容。现有的概念移除方法通常会引导图像生成过程,但可能会在无意中修改无关区域,从而导致与原始模型不一致。我们提出了一种在扩散模型中进行定向概念替换的新方法,能够移除特定概念而不影响非目标区域。我们的方法引入了一个专用的概念定位器,用于在去噪过程中精确识别目标概念;该定位器通过少样本学习训练,因此只需要极少的标注数据。在识别出的区域内,我们引入了无需训练的 Dual Prompts Cross-Attention(DPCA,双提示交叉注意力)模块来替换目标概念,从而确保对周围内容的干扰最小。我们从概念定位精度和替换效率两方面评估了该方法。实验结果表明,我们的方法在定位目标概念方面实现了更高的精度,并且能够执行连贯的概念替换,同时对非目标区域影响极小,性能优于现有方法。

英文摘要:

As large-scale diffusion models continue to advance, they excel at producing high-quality images but often generate unwanted content, such as sexually explicit or violent content. Existing methods for concept removal generally guide the image generation process but can unintentionally modify unrelated regions, leading to inconsistencies with the original model. We propose a novel approach for targeted concept replacing in diffusion models, enabling specific concepts to be removed without affecting non-target areas. Our method introduces a dedicated concept localizer for precisely identifying the target concept during the denoising process, trained with few-shot learning to require minimal labeled data. Within the identified region, we introduce a training-free Dual Prompts Cross-Attention (DPCA) module to substitute the target concept, ensuring minimal disruption to surrounding content. We evaluate our method on concept localization precision and replacement efficiency. Experimental results demonstrate that our method achieves superior precision in localizing target concepts and performs coherent concept replacement with minimal impact on non-target areas, outperforming existing approaches.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑