LoRAShop:基于校正流Transformer的无训练多概念图像生成与编辑
LoRAShop: Training-Free Multi-Concept Image Generation and Editing with Rectified Flow Transformers
AI总结:
LoRAShop是首个基于LoRA模型的多概念图像编辑框架,利用Flux风格扩散Transformer特征在去噪早期的空间激活特性推导解耦潜在掩码,实现无需重训练的多主体或风格无缝集成,在保留全局上下文的同时显著提升身份保持效果。
摘要:
我们提出了LoRAShop,这是首个使用LoRA模型进行多概念图像编辑的框架。LoRAShop基于对Flux风格扩散Transformer内部特征交互模式的关键观察:在去噪过程早期,特定概念的Transformer特征会在空间连贯的区域激活。我们利用这一观察,在先前的正向传播中为每个概念推导出解耦的潜在掩码,并仅在包含待个性化概念的区域内混合相应的LoRA权重。由此产生的编辑能够将多个主体或风格无缝集成到原始场景中,同时保留全局上下文、光照和精细细节。我们的实验表明,与基线相比,LoRAShop在身份保持方面表现更优。通过消除重新训练和外部约束,LoRAShop将个性化扩散模型转化为实用的“LoRA版Photoshop”工具,并为组合视觉叙事和快速创意迭代开辟了新途径。
英文摘要:
We introduce LoRAShop, the first framework for multi-concept image editing with LoRA models. LoRAShop builds on a key observation about the feature interaction patterns inside Flux-style diffusion transformers: concept-specific transformer features activate spatially coherent regions early in the denoising process. We harness this observation to derive a disentangled latent mask for each concept in a prior forward pass and blend the corresponding LoRA weights only within regions bounding the concepts to be personalized. The resulting edits seamlessly integrate multiple subjects or styles into the original scene while preserving global context, lighting, and fine details. Our experiments demonstrate that LoRAShop delivers better identity preservation compared to baselines. By eliminating retraining and external constraints, LoRAShop turns personalized diffusion models into a practical `photoshop-with-LoRAs' tool and opens new avenues for compositional visual storytelling and rapid creative iteration.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。