通过消除互信息实现无补偿的文本到图像扩散模型机器反学习
Compensation-free Machine Unlearning in Text-to-Image Diffusion Models by Eliminating the Mutual Information
AI总结：
本文提出无补偿的概念擦除方法MiM-MU，通过最小化互信息精准消除不需要的知识，从而在保持其他生成质量的同时无需事后补偿。
摘要：
扩散模型强大的生成能力引发了关于生成敏感或不想要内容的隐私和安全担忧。为此，机器反学习（MU）——在扩散模型中通常称为概念擦除（CE）——已被引入以从模型参数中移除特定知识的同时保持无辜知识。尽管近年来取得了进展，但现有反学习方法往往面临过度和无差别地移除的问题，导致无辜生成的质量显著下降。为了保持模型效用，先前的工作依赖于补偿，即重新吸收剩余数据的一个子集或显式限制与预训练模型在剩余概念上的分歧。然而，我们发现补偿范围之外的生成仍然受到影响，这表明此类事后补偿本质上不足以保持大规模生成模型的一般效用。因此，在本文中，我们倡导开发无补偿的概念擦除操作，这些操作能够精确识别并消除不需要的知识，使得对其他生成的影响最小。在技术上，我们提出MiM-MU，即通过最小化与概念的互信息来反学习，通过精心设计以提高计算效率，并保持其他概念的采样分布。广泛的评估显示，我们提出的方法在有效去除概念的同时保持了其他概念的高质量生成，并且值得注意的是，首次无需任何事后补偿。
英文摘要：
The powerful generative capabilities of diffusion models have raised growing privacy and safety concerns regarding generating sensitive or undesired content. In response, machine unlearning (MU) -- commonly referred to as concept erasure (CE) in diffusion models -- has been introduced to remove specific knowledge from model parameters meanwhile preserving innocent knowledge. Despite recent advancements, existing unlearning methods often suffer from excessive and indiscriminate removal, which leads to substantial degradation in the quality of innocent generations. To preserve model utility, prior works rely on compensation, i.e., re-assimilating a subset of the remaining data or explicitly constraining the divergence from the pre-trained model on remaining concepts. However, we reveal that generations beyond the compensation scope still suffer, suggesting such post-remedial compensations are inherently insufficient for preserving the general utility of large-scale generative models. Therefore, in this paper, we advocate for developing compensation-free concept erasure operations, which precisely identify and eliminate the undesired knowledge such that the impact on other generations is minimal. In technique, we propose to MiM-MU, which is to unlearn a concept by minimizing the mutual information with a delicate design for computational effectiveness and for maintaining sampling distribution for other concepts. Extensive evaluations demonstrate that our proposed method achieves effective concept removal meanwhile maintaining high-quality generations for other concepts, and remarkably, without relying on any post-remedial compensation for the first time.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
