面向说话人头部视频生成的在线尺度变换学习
Learning Online Scale Transformation for Talking Head Video Generation
AI总结：
针对单样本说话人头部视频生成中源图像与驱动图像尺度差异导致面部重演效果不佳的问题，提出带尺度变换模块的在线尺度变换面部重演网络，无需锚帧即可实现准确动作迁移，生成尺度准确的高质量跨身份重演人脸。
摘要：
单样本说话人头部视频生成利用一张源图像和驱动视频生成合成视频，使源人物的面部动作模仿驱动视频中的动作。然而，源图像与驱动图像之间的尺度差异仍是面部重演的一大挑战。现有方法试图在驱动视频中定位与源图像对齐效果最佳的帧，但对齐不精确会导致结果欠佳。\n 为此，我们提出一种尺度变换模块，该模块可利用源图像与驱动帧检测关键点中保留的尺度差异信息，自动调整驱动图像的尺度以适配源图像的尺度。此外，为了在生成过程中持续感知人脸的尺度信息，我们将从尺度变换模块学习到的尺度信息融入生成过程的每一层，以产出尺度准确的最终结果。通过所提出的在线尺度变换面部重演网络的作用，我们的方法无需任何锚帧即可在两幅图像之间实现准确的动作迁移。大量实验表明，我们提出的方法可根据源人脸自动调整驱动人脸的尺度，并在跨身份面部重演中生成尺度准确的高质量人脸。
英文摘要：
One-shot talking head video generation uses a source image and driving video to create a synthetic video where the source person's facial movements imitate those of the driving video. However, differences in scale between the source and driving images remain a challenge for face reenactment. Existing methods attempt to locate a frame in the driving video that aligns best with the source image, but imprecise alignment can result in suboptimal outcomes. To this end, we introduce a scale transformation module that can automatically adjust the scale of the driving image to fit that of the source image, by using the information of scale difference maintained in the detected keypoints of the source image and the driving frame. Furthermore, to keep perceiving the scale information of faces during the generation process, we incorporate the scale information learned from the scale transformation module into each layer of the generation process to produce a final result with an accurate scale. Our method can perform accurate motion transfer between the two images without any anchor frame, achieved through the contributions of the proposed online scale transformation facial reenactment network. Extensive experiments have demonstrated that our proposed method adjusts the scale of the driving face automatically according to the source face, and generates high-quality faces with an accurate scale in the cross-identity facial reenactment.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
