G3FA：用于人脸动画的几何引导GAN
G3FA: Geometry-guided GAN for Face Animation
AI总结：
本文提出G3FA框架，通过融合逆向渲染提取3D几何信息、判别器加权平均集成及2D运动扭曲与体绘制技术，提升了仅基于2D图像的实时人脸重演的几何一致性与生成质量。
摘要：
人脸图像动画化旨在以自然逼真的方式合成所需的源身份，同时模仿驱动视频中的人脸动作。在此背景下，生成对抗网络在利用单张源图像进行实时人脸重演方面展现出了卓越的潜力，但与基于图形学的方法相比，受限于有限的几何一致性。在本文中，我们引入了用于人脸动画的几何引导GAN（G3FA）来解决这一局限性。我们的新方法使人脸动画模型能够仅使用2D图像即可融合3D信息，从而提升了说话头部合成模型的图像生成能力。我们融合了逆向渲染技术以提取3D人脸几何属性，通过判别器的加权平均集成改善了向生成器的反馈回路。在我们的人脸重演模型中，我们利用2D运动扭曲来捕捉运动动态，并结合正交射线采样和体绘制技术来生成最终的视觉输出。为了评估G3FA的性能，我们在VoxCeleb2和TalkingHead基准上使用各种评估协议进行了全面实验，以证明我们提出的框架与最先进的实时人脸动画方法相比的有效性。
英文摘要：
Animating human face images aims to synthesize a desired source identity in a natural-looking way mimicking a driving video's facial movements. In this context, Generative Adversarial Networks have demonstrated remarkable potential in real-time face reenactment using a single source image, yet are constrained by limited geometry consistency compared to graphic-based approaches. In this paper, we introduce Geometry-guided GAN for Face Animation (G3FA) to tackle this limitation. Our novel approach empowers the face animation model to incorporate 3D information using only 2D images, improving the image generation capabilities of the talking head synthesis model. We integrate inverse rendering techniques to extract 3D facial geometry properties, improving the feedback loop to the generator through a weighted average ensemble of discriminators. In our face reenactment model, we leverage 2D motion warping to capture motion dynamics along with orthogonal ray sampling and volume rendering techniques to produce the ultimate visual output. To evaluate the performance of our G3FA, we conducted comprehensive experiments using various evaluation protocols on VoxCeleb2 and TalkingHead benchmarks to demonstrate the effectiveness of our proposed framework compared to the state-of-the-art real-time face animation methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
