EZIGen：通过精确的主体编码与解耦引导增强零样本个性化图像生成
EZIGen: Enhancing zero-shot personalized image generation with precise subject encoding and decoupled guidance
AI总结：
本文提出 EZIGen 方法，通过利用预训练 Diffusion UNet 作为主体编码器并分离文本与主体引导的去噪阶段，有效解决了零样本个性化图像生成中的细节丢失与引导不平衡问题，在减少训练数据的同时实现了最先进的生成性能。
摘要：
零样本个性化图像生成模型旨在生成同时符合给定文本提示词和主体图像的图像，这要求模型融合这两种引导源。现有方法通常难以捕捉细粒度的主体细节，且经常优先考虑其中一种引导而忽视另一种，导致主体编码效果不佳和生成不平衡。在本研究中，我们揭示了克服这些缺陷的关键见解，具体而言：1) 主体图像编码器的选择对主体身份保持和训练效率有至关重要的影响；2) 文本和主体引导应在不同的去噪阶段生效。基于这些见解，我们引入了一种新方法 EZIGen，它包含两个主要组件：利用固定的预训练 Diffusion UNet 本身作为主体编码器，并遵循一个通过分离两者的主导阶段并重访某些时间步来平衡这两种引导、从而引导主体迁移质量的过程。通过这两个组件，最初基于 SD2.1-base 构建的 EZIGen 在使用统一模型且训练数据量减少 100 倍的情况下，在多个个性化生成基准上取得了最先进的性能。此外，通过进一步将我们的设计迁移到 SDXL，证明了 EZIGen 是一种用于个性化生成的通用且与模型无关的解决方案。
英文摘要：
Zero-shot personalized image generation models aim to produce images that align with both a given text prompt and subject image, requiring the model to incorporate both sources of guidance. Existing methods often struggle to capture fine-grained subject details and frequently prioritize one form of guidance over the other, resulting in suboptimal subject encoding and imbalanced generation. In this study, we uncover key insights into overcoming such drawbacks, notably that 1) the choice of the subject image encoder critically influences subject identity preservation and training efficiency, and 2) the text and subject guidance should take effect at different denoising stages. Building on these insights, we introduce a new approach, EZIGen, that employs two main components: leveraging a fixed pre-trained Diffusion UNet itself as subject encoder, following a process that balances the two guidances by separating their dominance stage and revisiting certain time steps to bootstrap subject transfer quality. Through these two components, EZIGen, initially built upon SD2.1-base, achieved state-of-the-art performances on multiple personalized generation benchmarks with a unified model, while using 100 times less training data. Moreover, by further migrating our design to SDXL, EZIGen is proven to be a versatile model-agnostic solution for personalized generation. Demo Page: zichengduan.github.io/pages/EZIGen/index.html
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
