HyperReenact：通过联合学习人脸优化与重定向实现单样本人脸重演
HyperReenact: One-Shot Reenactment via Jointly Learning to Refine and Retarget Faces
AI总结：
针对现有单样本人脸重演易产生伪影、极端姿态下效果差的问题，提出HyperReenact方法，借助预训练StyleGAN2和超网络实现身份优化与姿态重定向，在标准基准上取得更优的无伪影生成效果与鲁棒性。
摘要：
本文提出了一种名为HyperReenact的神经人脸重演方法，旨在生成由目标面部姿态驱动的、具有源身份特征的逼真说话人头部图像。现有最先进的人脸重演方法通过训练可控生成模型来合成逼真的人脸图像，但生成的重演人脸往往存在明显的视觉伪影，尤其是在头部姿态发生极端变化的挑战性条件下；或是需要成本高昂的少样本微调才能更好地保留源身份特征。为解决这些局限，我们利用预训练StyleGAN2生成器的照片级真实生成能力和解耦特性，先将真实图像逆映射到其潜在空间，再使用超网络执行两项任务：（i）源身份特征优化，以及（ii）面部姿态重定向，从而摆脱了对通常会产生伪影的外部编辑方法的依赖。我们的方法在单样本设置下运行（即仅使用单个源帧），支持跨主体重演，无需任何针对特定主体的微调。我们在VoxCeleb1和VoxCeleb2标准基准上，将该方法与多种最先进技术进行了定量与定性比较，结果表明我们的方法在生成无伪影图像方面更具优势，即使在头部姿态极端变化的情况下也表现出显著的鲁棒性。我们已将代码和预训练模型公开于：https://github.com/StelaBou/HyperReenact 。
英文摘要：
In this paper, we present our method for neural face reenactment, called HyperReenact, that aims to generate realistic talking head images of a source identity, driven by a target facial pose. Existing state-of-the-art face reenactment methods train controllable generative models that learn to synthesize realistic facial images, yet producing reenacted faces that are prone to significant visual artifacts, especially under the challenging condition of extreme head pose changes, or requiring expensive few-shot fine-tuning to better preserve the source identity characteristics. We propose to address these limitations by leveraging the photorealistic generation ability and the disentangled properties of a pretrained StyleGAN2 generator, by first inverting the real images into its latent space and then using a hypernetwork to perform: (i) refinement of the source identity characteristics and (ii) facial pose re-targeting, eliminating this way the dependence on external editing methods that typically produce artifacts. Our method operates under the one-shot setting (i.e., using a single source frame) and allows for cross-subject reenactment, without requiring any subject-specific fine-tuning. We compare our method both quantitatively and qualitatively against several state-of-the-art techniques on the standard benchmarks of VoxCeleb1 and VoxCeleb2, demonstrating the superiority of our approach in producing artifact-free images, exhibiting remarkable robustness even under extreme head pose changes. We make the code and the pretrained models publicly available at: https://github.com/StelaBou/HyperReenact .
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
