FPGA：灵活的肖像生成方法
FPGA: Flexible Portrait Generation Approach
AI总结：
针对肖像生成中低分辨率人脸和多身份控制难题，提出FPGA系统，含MMF训练策略和DIIR推理框架，并构建IDZoom数据集，实现高效可控生成，推理速度在L20上低于2.5秒。
摘要：
肖像保真生成是生成模型中的一个重要研究领域。当前方法在生成低分辨率人脸的全身体图像时面临挑战，尤其是在多身份照片现象中。为解决这些问题，我们提出了一个名为FPGA的综合系统，并构建了一个百万级的多模态数据集IDZoom用于训练。FPGA由多模态融合训练策略（MMF）和基于DDIM反演的身份恢复推理框架（DIIR）组成。MMF旨在在指定面部区域激活指定身份。DIIR旨在解决人脸伪影问题，同时保持背景不变。此外，DIIR是即插即用的，可应用于任何基于扩散的肖像生成方法以增强其性能。DIIR还适用于换脸任务，并且也适用于风格化人脸。为验证FPGA的有效性，我们进行了广泛的对比和消融实验。实验结果表明，FPGA在主观和客观指标上均具有显著优势，并在多身份场景中实现了可控生成。此外，我们主要基于精心设计的重参数化方法RepControlNet，将推理速度加速至单张L20显卡上2.5秒以内。
英文摘要：
Portrait Fidelity Generation is a prominent research area in generative models.Current methods face challenges in generating full-body images with low-resolution faces, especially in multi-ID photo phenomenon.To tackle these issues, we propose a comprehensive system called FPGA and construct a million-level multi-modal dataset IDZoom for training.FPGA consists of Multi-Mode Fusion training strategy (MMF) and DDIM Inversion based ID Restoration inference framework (DIIR). The MMF aims to activate the specified ID in the specified facial region. The DIIR aims to address the issue of face artifacts while keeping the background.Furthermore, DIIR is plug-and-play and can be applied to any diffusion-based portrait generation method to enhance their performance. DIIR is also capable of performing face-swapping tasks and is applicable to stylized faces as well.To validate the effectiveness of FPGA, we conducted extensive comparative and ablation experiments. The experimental results demonstrate that FPGA has significant advantages in both subjective and objective metrics, and achieves controllable generation in multi-ID scenarios. In addition, we accelerate the inference speed to within 2.5 seconds on a single L20 graphics card mainly based on our well designed reparameterization method, RepControlNet.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
