挖掘你自己的秘密：用于文本到图像扩散模型持续个性化的扩散分类器分数
Mining Your Own Secrets: Diffusion Classifier Scores for Continual Personalization of Text-to-Image Diffusion Models
AI总结：
本文提出利用扩散分类器分数对文本到图像扩散模型的参数空间和函数空间进行正则化，以在无回放和低秩适配器条件下实现零存储与参数开销的持续个性化，并超越C-LoRA等现有方法。
摘要：
个性化文本到图像扩散模型因其能够根据用户定义的文本描述和少量图像高效获取新概念而日益流行。然而，在现实世界中，用户可能希望在一个模型上逐个个性化多个概念，但由于存储或隐私方面的考虑，无法访问先前概念的数据。面对这种持续学习（CL）设置，大多数个性化方法难以在获取新概念和保留先前概念之间找到平衡——这是持续个性化（CP）旨在解决的挑战。受依赖类别特定信息进行正则化的成功持续学习方法的启发，我们利用固有的类别条件密度估计（也称为扩散分类器（DC）分数）来实现文本到图像扩散模型的持续个性化。具体而言，我们提出使用DC分数对文本到图像扩散模型的参数空间和函数空间进行正则化，以实现持续个性化。通过使用多种不同的评估设置、数据集和指标，我们表明我们提出的基于正则化的持续个性化方法优于最先进的C-LoRA以及其他基线方法。最后，通过在无回放的持续学习设置下运行并基于低秩适配器，与最先进方法相比，我们的方法分别实现了零存储开销和零参数开销。我们的项目页面：https://srvcodes.github.io/continual_personalization/
英文摘要：
Personalized text-to-image diffusion models have grown popular for their ability to efficiently acquire a new concept from user-defined text descriptions and a few images. However, in the real world, a user may wish to personalize a model on multiple concepts but one at a time, with no access to the data from previous concepts due to storage/privacy concerns. When faced with this continual learning (CL) setup, most personalization methods fail to find a balance between acquiring new concepts and retaining previous ones -- a challenge that continual personalization (CP) aims to solve. Inspired by the successful CL methods that rely on class-specific information for regularization, we resort to the inherent class-conditioned density estimates, also known as diffusion classifier (DC) scores, for continual personalization of text-to-image diffusion models. Namely, we propose using DC scores for regularizing the parameter-space and function-space of text-to-image diffusion models, to achieve continual personalization. Using several diverse evaluation setups, datasets, and metrics, we show that our proposed regularization-based CP methods outperform the state-of-the-art C-LoRA, and other baselines. Finally, by operating in the replay-free CL setup and on low-rank adapters, our method incurs zero storage and parameter overhead, respectively, over the state-of-the-art. Our project page: https://srvcodes.github.io/continual_personalization/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
