PaRa：通过参数秩降低实现文本到图像扩散的个性化
PaRa: Personalizing Text-to-Image Diffusion via Parameter Rank Reduction
AI总结：
本文提出 PaRa 方法，通过降低扩散模型参数秩来约束生成空间，在保持文本可编辑性的同时实现高效个性化，相比 LoRA 参数效率提升 2 倍且目标对齐更好。
摘要：
个性化一个大规模预训练的文本到图像（T2I）扩散模型具有挑战性，因为它通常难以在其训练数据分布与目标分布之间做出适当的权衡，即仅用少量目标图像学习一个新概念以实现个性化（与个性化目标对齐），同时保持文本可编辑性（与多样化的文本提示对齐）。在本文中，我们提出了 PaRa，一种有效且高效的参数秩降低方法，用于 T2I 模型个性化，通过显式控制扩散模型参数的秩，将其初始的多样化生成空间限制到一个较小且平衡良好的目标空间。我们的设计动机源于这样一个事实：将 T2I 模型驯化到一个新概念（如特定艺术风格）意味着一个较小的生成空间。为此，通过在微调过程中降低模型参数的秩，我们可以有效地将去噪采样轨迹的空间约束到目标上。通过全面的实验，我们表明 PaRa 在单主体/多主体生成以及单图像编辑方面比现有微调方法具有显著优势。值得注意的是，与主流的微调技术 LoRA 相比，PaRa 实现了更好的参数效率（可学习参数少 2 倍）和更好的目标图像对齐。
英文摘要：
Personalizing a large-scale pretrained Text-to-Image (T2I) diffusion model is challenging as it typically struggles to make an appropriate trade-off between its training data distribution and the target distribution, i.e., learning a novel concept with only a few target images to achieve personalization (aligning with the personalized target) while preserving text editability (aligning with diverse text prompts). In this paper, we propose PaRa, an effective and efficient Parameter Rank Reduction approach for T2I model personalization by explicitly controlling the rank of the diffusion model parameters to restrict its initial diverse generation space into a small and well-balanced target space. Our design is motivated by the fact that taming a T2I model toward a novel concept such as a specific art style implies a small generation space. To this end, by reducing the rank of model parameters during finetuning, we can effectively constrain the space of the denoising sampling trajectories towards the target. With comprehensive experiments, we show that PaRa achieves great advantages over existing finetuning approaches on single/multi-subject generation as well as single-image editing. Notably, compared to the prevailing fine-tuning technique LoRA, PaRa achieves better parameter efficiency (2x fewer learnable parameters) and much better target image alignment.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
