FreeTuner:任意主体任意风格的无训练扩散
FreeTuner: Any Subject in Any Style with Training-free Diffusion
AI总结:
针对组合个性化中概念纠缠等问题,提出无需训练的FreeTuner方法,通过两阶段解耦策略和风格引导,实现任意主体与任意风格的组合生成,实验验证其有效性。
摘要:
随着扩散模型的进步,各种个性化图像生成方法已被提出。然而,几乎所有现有工作只关注于主体驱动或风格驱动的个性化。同时,最先进的方法在实现组合个性化(即组合不同的主体和风格概念)时面临若干挑战,例如概念解耦、统一重建范式和训练数据不足。为解决这些问题,我们引入了FreeTuner,一种灵活且无需训练的组合个性化方法,能够生成任意用户提供的主体在任意用户提供的风格下的图像(见图1)。我们的方法采用一种解耦策略,将生成过程分为两个阶段,以有效缓解概念纠缠。FreeTuner利用扩散模型中的中间特征进行主体概念表示,并引入风格引导来使合成图像与风格概念对齐,确保同时保留主体的结构和风格的美学特征。大量实验证明了FreeTuner在各种个性化设置下的生成能力。
英文摘要:
With the advance of diffusion models, various personalized image generation methods have been proposed. However, almost all existing work only focuses on either subject-driven or style-driven personalization. Meanwhile, state-of-the-art methods face several challenges in realizing compositional personalization, i.e., composing different subject and style concepts, such as concept disentanglement, unified reconstruction paradigm, and insufficient training data. To address these issues, we introduce FreeTuner, a flexible and training-free method for compositional personalization that can generate any user-provided subject in any user-provided style (see Figure 1). Our approach employs a disentanglement strategy that separates the generation process into two stages to effectively mitigate concept entanglement. FreeTuner leverages the intermediate features within the diffusion model for subject concept representation and introduces style guidance to align the synthesized images with the style concept, ensuring the preservation of both the subject's structure and the style's aesthetic features. Extensive experiments have demonstrated the generation ability of FreeTuner across various personalization settings.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。