LatRef-Diff: 基于潜在和参考引导的扩散模型用于面部属性编辑和风格操控
LatRef-Diff: Latent and Reference-Guided Diffusion for Facial Attribute Editing and Style Manipulation
AI总结：
本文提出LatRef-Diff，通过潜在和参考引导方法改进面部属性编辑和风格操控，解决传统扩散模型在属性控制和风格操控中的不足，采用风格代码和模块化设计提升精度和图像质量。
摘要：
面部属性编辑和风格操控对于虚拟化身和照片编辑等应用至关重要。然而，由于面部结构的复杂性和属性间的强相关性，实现精确控制而不影响无关特征具有挑战性。尽管条件GANs取得进展，但存在准确性和训练稳定性问题。扩散模型虽具潜力，但风格操控受限于语义方向的表达能力有限。本文提出LatRef-Diff，一种新型扩散框架，通过替换传统语义方向为风格代码，并提出两种生成方法：潜在和参考引导。基于这些风格代码，设计风格调制模块，将它们整合到目标图像中，实现随机和定制化的风格操控。该模块结合可学习向量、交叉注意力机制和分层设计以提高精度和图像质量。此外，为增强训练稳定性并消除对配对图像（如编辑前后的图像）的依赖，提出前向-后向一致性训练策略。该策略首先使用图像特定的语义方向近似移除目标属性，然后通过风格调制恢复，通过感知和分类损失进行引导。在CelebA-HQ上的广泛实验表明，LatRef-Diff在定性和定量评估中均达到最先进的性能。消融研究验证了模型设计选择的有效性。
英文摘要：
Facial attribute editing and style manipulation are crucial for applications like virtual avatars and photo editing. However, achieving precise control over facial attributes without altering unrelated features is challenging due to the complexity of facial structures and the strong correlations between attributes. While conditional GANs have shown progress, they are limited by accuracy issues and training instability. Diffusion models, though promising, face challenges in style manipulation due to the limited expressiveness of semantic directions. In this paper, we propose LatRef-Diff, a novel diffusion-based framework that addresses these limitations. We replace the traditional semantic directions in diffusion models with style codes and propose two methods for generating them: latent and reference guidance. Based on these style codes, we design a style modulation module that integrates them into the target image, enabling both random and customized style manipulation. This module incorporates learnable vectors, cross-attention mechanisms, and a hierarchical design to improve accuracy and image quality. Additionally, to enhance training stability while eliminating the need for paired images (e.g., before and after editing), we propose a forward-backward consistency training strategy. This strategy first removes the target attribute approximately using image-specific semantic directions and then restores it via style modulation, guided by perceptual and classification losses. Extensive experiments on CelebA-HQ demonstrate that LatRef-Diff achieves state-of-the-art performance in both qualitative and quantitative evaluations. Ablation studies validate the effectiveness of our model's design choices.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
