PIDiff：使用扩散模型实现个性化身份的图像定制
PIDiff: Image Customization for Personalized Identities with Diffusion Models
AI总结：
针对个性化身份生成中身份与背景语义纠缠导致特征丢失和多样性下降的问题，本文提出 PIDiff，结合 W+ 空间、身份定制微调、交叉注意力块与参数优化，实现准确定位并保持生成多样性。
摘要：
面向个性化身份的文本到图像生成旨在利用文本提示和身份图像将特定身份融入图像。基于 DDPM 强大的生成能力，以往许多工作采用额外提示（如文本嵌入和 CLIP 图像嵌入）来表示身份信息，但它们未能解耦身份信息与背景信息。因此，生成的图像不仅会丢失关键身份特征，还会出现多样性显著下降的问题。为解决这一问题，已有工作将 StyleGAN 的 W+ 空间与扩散模型结合，利用该空间通过多级特征提取提供更准确、更全面的身份特征表示。然而，训练过程中野外图像内身份与背景信息的纠缠阻碍了准确的身份定位，导致身份与背景之间严重的语义干扰。本文提出一种新颖的、基于微调的扩散模型 PIDiff，用于个性化身份的文本到图像生成；该模型利用 W+ 空间和身份定制的微调策略避免语义纠缠，并实现准确的特征提取与定位。PIDiff 还可通过保留 W+ 空间中由粗到细变化的身份特征特性来实现风格编辑。通过结合所提出的交叉注意力块和参数优化策略，PIDiff 在推理时保留身份信息，并维持预训练模型对野外图像的生成能力。实验结果验证了本文方法在该任务上的有效性。
英文摘要：
Text-to-image generation for personalized identities aims at incorporating the specific identity into images using a text prompt and an identity image. Based on the powerful generative capabilities of DDPMs, many previous works adopt additional prompts, such as text embeddings and CLIP image embeddings, to represent the identity information, while they fail to disentangle the identity information and background information. As a result, the generated images not only lose key identity characteristics but also suffer from significantly reduced diversity. To address this issue, previous works have combined the W+ space from StyleGAN with diffusion models, leveraging this space to provide a more accurate and comprehensive representation of identity features through multi-level feature extraction. However, the entanglement of identity and background information in in-the-wild images during training prevents accurate identity localization, resulting in severe semantic interference between identity and background. In this paper, we propose a novel fine-tuning-based diffusion model for personalized identities text-to-image generation, named PIDiff, which leverages the W+ space and an identity-tailored fine-tuning strategy to avoid semantic entanglement and achieves accurate feature extraction and localization. Style editing can also be achieved by PIDiff through preserving the characteristics of identity features in the W+ space, which vary from coarse to fine. Through the combination of the proposed cross-attention block and parameter optimization strategy, PIDiff preserves the identity information and maintains the generation capability for in-the-wild images of the pre-trained model during inference. Our experimental results validate the effectiveness of our method in this task.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
