arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新，休息一下吧，周末愉快~~
arXiv 2401.07519cs.CVcs.AI
收藏

InstantID：数秒内实现的零样本身份保持生成

InstantID: Zero-shot Identity-Preserving Generation in Seconds

Xiaohongshu Inc（小红书公司）；Peking University（北京大学）

Qixun Wang, Xu Bai, Haofan Wang, Zekui Qin, Anthony Chen, Huaxia Li, Xu Tang, Yao Hu

备注：
Technical Report, project page available at https://instantid.github.io/
阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对现有个性化图像合成方法存在的存储需求高、微调耗时长、人脸保真度不足等问题，研究提出基于扩散模型的即插即用方案InstantID，通过IdentityNet融合人脸、关键点与文本提示，实现单图秒级高保真零样本身份保持生成，可兼容主流预训练扩散模型。

摘要：

个性化图像合成领域已取得显著进展，代表性方法包括Textual Inversion（文本反转）、DreamBooth和LoRA（低秩适配）。然而，这类方法的实际应用受限于高存储需求、漫长的微调过程，且需要多张参考图像。相比之下，现有基于ID嵌入的方法虽仅需单次前向推理，但仍面临诸多挑战：要么需要对大量模型参数进行广泛微调，要么与社区预训练模型不兼容，或是无法维持高人脸保真度。针对这些局限，我们提出了InstantID——一种强大的基于扩散模型的解决方案。我们的即插即用模块仅需单张人脸图像，就能灵活实现多种风格的图像个性化，同时确保高保真度。为实现这一目标，我们设计了全新的IdentityNet，通过施加强语义条件和弱空间条件，将人脸图像、关键点图像与文本提示相结合，以引导图像生成过程。InstantID展现出卓越的性能与效率，在身份保持至关重要的实际应用中极具价值。此外，我们的工作可与SD1.5、SDXL等主流预训练文本到图像扩散模型无缝集成，作为适配性极强的插件使用。我们的代码和预训练检查点将发布于https://github.com/InstantID/InstantID。

英文摘要：

There has been significant progress in personalized image synthesis with methods such as Textual Inversion, DreamBooth, and LoRA. Yet, their real-world applicability is hindered by high storage demands, lengthy fine-tuning processes, and the need for multiple reference images. Conversely, existing ID embedding-based methods, while requiring only a single forward inference, face challenges: they either necessitate extensive fine-tuning across numerous model parameters, lack compatibility with community pre-trained models, or fail to maintain high face fidelity. Addressing these limitations, we introduce InstantID, a powerful diffusion model-based solution. Our plug-and-play module adeptly handles image personalization in various styles using just a single facial image, while ensuring high fidelity. To achieve this, we design a novel IdentityNet by imposing strong semantic and weak spatial conditions, integrating facial and landmark images with textual prompts to steer the image generation. InstantID demonstrates exceptional performance and efficiency, proving highly beneficial in real-world applications where identity preservation is paramount. Moreover, our work seamlessly integrates with popular pre-trained text-to-image diffusion models like SD1.5 and SDXL, serving as an adaptable plugin. Our codes and pre-trained checkpoints will be available at https://github.com/InstantID/InstantID.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑