TruePose：面向全身份保持的姿态迁移的人体解析引导注意力扩散
TruePose: Human-Parsing-guided Attention Diffusion for Full-ID Preserving Pose Transfer
AI总结：
TruePose提出人体解析引导的注意力扩散方法，通过孪生UNet、HPFA和CAA模块在姿态迁移中同时保持面部与服装外观，实验证明其优于13种基线方法。
摘要：
姿态引导的人物图像合成（PGPIS）旨在生成既保持源图像中人物身份、又采用指定目标姿态（如骨架）的图像。尽管基于扩散的PGPIS方法在姿态变换过程中能有效保留面部特征，但它们往往难以在整个扩散过程中准确保持源图像中的服装细节。当源姿态与目标姿态差异较大时，这一局限性尤为突出，严重影响PGPIS在时尚行业中的应用，因为服装风格保持对版权保护至关重要。我们的分析表明，该局限性主要源于条件扩散模型的注意力模块未能充分捕获和保持服装图案。为解决这一问题，我们提出了人体解析引导的注意力扩散方法，这是一种在生成高质量结果的同时有效保持面部和服装外观的新方法。我们提出了一种人体解析感知的孪生网络，由三个关键组件组成：双相同UNet（用于扩散去噪的TargetNet和用于源图像嵌入提取的SourceNet）、人体解析引导的融合注意力（HPFA）以及CLIP引导的注意力对齐（CAA）。HPFA和CAA模块能够自适应且有效地将面部和服装图案嵌入到目标图像生成过程中。在店铺服装检索基准和最新的自然场景人体编辑数据集上的大量实验表明，我们的方法在保持源图像中面部和服装外观方面，相较于13种基线方法具有显著优势。
英文摘要：
Pose-Guided Person Image Synthesis (PGPIS) generates images that maintain a subject's identity from a source image while adopting a specified target pose (e.g., skeleton). While diffusion-based PGPIS methods effectively preserve facial features during pose transformation, they often struggle to accurately maintain clothing details from the source image throughout the diffusion process. This limitation becomes particularly problematic when there is a substantial difference between the source and target poses, significantly impacting PGPIS applications in the fashion industry where clothing style preservation is crucial for copyright protection. Our analysis reveals that this limitation primarily stems from the conditional diffusion model's attention modules failing to adequately capture and preserve clothing patterns. To address this limitation, we propose human-parsing-guided attention diffusion, a novel approach that effectively preserves both facial and clothing appearance while generating high-quality results. We propose a human-parsing-aware Siamese network that consists of three key components: dual identical UNets (TargetNet for diffusion denoising and SourceNet for source image embedding extraction), a human-parsing-guided fusion attention (HPFA), and a CLIP-guided attention alignment (CAA). The HPFA and CAA modules can embed the face and clothes patterns into the target image generation adaptively and effectively. Extensive experiments on both the in-shop clothes retrieval benchmark and the latest in-the-wild human editing dataset demonstrate our method's significant advantages over 13 baseline approaches for preserving both facial and clothes appearance in the source image.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
