SelfieAvatar: 从自拍视频实现实时头部虚拟形象重现
SelfieAvatar: Real-time Head Avatar reenactment from a Selfie Video
AI总结：
本研究提出基于3DMM和StyleGAN的方法，利用自拍视频实现实时头部虚拟形象重现，提升细节还原与高保真度。
摘要：
头部虚拟形象重现旨在从单目视频中创建可动画化的个人虚拟形象，是用于社交信号理解、游戏、人机交互和计算机视觉等应用的基础要素。最近基于3D可变形模型（3DMM）的面部重建方法在高保真面部估计方面取得了显著成果。然而一方面，它们难以在实时情况下捕捉整个头部，包括非面部区域和背景细节，这是产生逼真、高保真的头部虚拟形象的关键要素。另一方面，利用生成对抗网络（GANs）从视频生成头部虚拟形象的最新方法可以实现高质量的重现，但存在难以再现细颗粒度头部细节（如皱纹和头发纹理）的局限性。此外，现有方法通常依赖大量的训练数据，很少专注于仅使用简单的自拍视频来实现虚拟形象重现。为了解决这些挑战，本研究提出了一种利用自拍视频进行详细头部虚拟形象重现的方法。该方法结合了3DMM与基于StyleGAN的生成器。提出了一种详细的重建模型，结合了混合损失函数，用于对抗训练过程中的前景重建和虚拟形象图像生成，以恢复高频细节。在自我重现和跨重现任务上的定性与定量评估表明，所提出的方法在丰富而复杂的纹理方面实现了优于现有方法的头部虚拟形象重建。
英文摘要：
Head avatar reenactment focuses on creating animatable personal avatars from monocular videos, serving as a foundational element for applications like social signal understanding, gaming, human-machine interaction, and computer vision. Recent advances in 3D Morphable Model (3DMM)-based facial reconstruction methods have achieved remarkable high-fidelity face estimation. However, on the one hand, they struggle to capture the entire head, including non-facial regions and background details in real time, which is an essential aspect for producing realistic, high-fidelity head avatars. On the other hand, recent approaches leveraging generative adversarial networks (GANs) for head avatar generation from videos can achieve high-quality reenactments but encounter limitations in reproducing fine-grained head details, such as wrinkles and hair textures. In addition, existing methods generally rely on a large amount of training data, and rarely focus on using only a simple selfie video to achieve avatar reenactment. To address these challenges, this study introduces a method for detailed head avatar reenactment using a selfie video. The approach combines 3DMMs with a StyleGAN-based generator. A detailed reconstruction model is proposed, incorporating mixed loss functions for foreground reconstruction and avatar image generation during adversarial training to recover high-frequency details. Qualitative and quantitative evaluations on self-reenactment and cross-reenactment tasks demonstrate that the proposed method achieves superior head avatar reconstruction with rich and intricate textures compared to existing approaches.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
