arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2406.02366cs.LGcs.AI
收藏

Finding NeMo:定位扩散模型中负责记忆的神经元

Finding NeMo: Localizing Neurons Responsible For Memorization in Diffusion Models

German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心);Technical University of Darmstadt(达姆施塔特工业大学);Hessian Center for AI (Hessian.AI)(黑森人工智能中心);Centre for Cognitive Science, Technical University of Darmstadt(达姆施塔特工业大学认知科学中心);CISPA Helmholtz Center for Information Security(CISPA亥姆霍兹信息安全中心)

Dominik Hintersdorf, Lukas Struppek, Kristian Kersting, Adam Dziedzic, Franziska Boenisch

AI总结:

针对扩散模型易记忆并复现训练数据引发的隐私与版权问题,提出NeMo方法定位交叉注意力层中负责记忆的单个神经元,失活后可避免数据复现、提升生成多样性,助力扩散模型负责任部署。

摘要:

扩散模型(DMs)能够生成细节极为丰富的高质量图像,其强大能力源于对海量数据的大规模训练——这些数据通常从网络抓取,未获得内容创作者的适当署名或同意。遗憾的是,这种做法引发了隐私和知识产权方面的担忧,因为DMs可能会记忆训练数据中潜在敏感或受版权保护的图像,并在推理阶段复现这些内容。\n此前解决该问题的思路分为两类:一类是修改扩散过程的输入,阻止DM在推理时生成记忆的样本;另一类是直接从训练数据中移除被记忆的内容。这些方案在DM处于安全且持续监控的环境中开发和部署时是可行的,但存在被攻击者绕过防护的风险,且当DM本身被公开发布时便不再有效。\n为解决这一问题,我们提出了NeMo,这是首个能将单个数据样本的记忆定位精确到DMs交叉注意力层神经元级别的方法。实验中我们发现了一个有趣的现象:在很多情况下,单个神经元就负责记忆特定的训练样本。通过失活这些记忆神经元,我们可以避免推理阶段复现训练数据,提升生成输出的多样性,同时缓解隐私和受版权保护数据的泄露问题。综上,NeMo为DMs的更负责任部署提供了助力。

英文摘要:

Diffusion models (DMs) produce very detailed and high-quality images. Their power results from extensive training on large amounts of data, usually scraped from the internet without proper attribution or consent from content creators. Unfortunately, this practice raises privacy and intellectual property concerns, as DMs can memorize and later reproduce their potentially sensitive or copyrighted training images at inference time. Prior efforts prevent this issue by either changing the input to the diffusion process, thereby preventing the DM from generating memorized samples during inference, or removing the memorized data from training altogether. While those are viable solutions when the DM is developed and deployed in a secure and constantly monitored environment, they hold the risk of adversaries circumventing the safeguards and are not effective when the DM itself is publicly released. To solve the problem, we introduce NeMo, the first method to localize memorization of individual data samples down to the level of neurons in DMs' cross-attention layers. Through our experiments, we make the intriguing finding that in many cases, single neurons are responsible for memorizing particular training samples. By deactivating these memorization neurons, we can avoid the replication of training data at inference time, increase the diversity in the generated outputs, and mitigate the leakage of private and copyrighted data. In this way, our NeMo contributes to a more responsible deployment of DMs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑