FSRT：基于分解外观、头部姿态和面部表情特征的人脸重现面部场景表示Transformer
FSRT: Facial Scene Representation Transformer for Face Reenactment from Factorized Appearance, Head-pose, and Facial Expression Features
AI总结：
本文提出FSRT，一种基于Transformer的人脸重现方法，通过自监督分解外观、头部姿态和表情特征，支持跨人重现和多源图像适应，实验表明其在运动迁移质量和时间一致性上优于现有方法。
摘要：
人脸重现任务是将驱动视频中的头部运动和面部表情迁移到源图像的外观上，源图像可能属于不同的人（跨人重现）。现有方法大多基于CNN，从源图像到当前驱动帧估计光流，然后进行修复和细化以生成输出动画。我们提出了一种基于Transformer的编码器，用于计算源图像的集合潜在表示。然后，我们使用基于Transformer的解码器预测查询像素的输出颜色，该解码器以从驱动帧中提取的关键点和面部表情向量为条件。源人物的潜在表示以自监督方式学习，分解其外观、头部姿态和面部表情。因此，它们非常适合跨人重现。与大多数相关工作相比，我们的方法自然地扩展到多个源图像，从而能够适应特定人物的面部动态。我们还提出了必要的数据增强和正则化方案，以防止过拟合并支持所学表示的泛化能力。我们在一项随机用户研究中评估了我们的方法。结果表明，在运动迁移质量和时间一致性方面，与最先进的方法相比具有优越性能。
英文摘要：
The task of face reenactment is to transfer the head motion and facial expressions from a driving video to the appearance of a source image, which may be of a different person (cross-reenactment). Most existing methods are CNN-based and estimate optical flow from the source image to the current driving frame, which is then inpainted and refined to produce the output animation. We propose a transformer-based encoder for computing a set-latent representation of the source image(s). We then predict the output color of a query pixel using a transformer-based decoder, which is conditioned with keypoints and a facial expression vector extracted from the driving frame. Latent representations of the source person are learned in a self-supervised manner that factorize their appearance, head pose, and facial expressions. Thus, they are perfectly suited for cross-reenactment. In contrast to most related work, our method naturally extends to multiple source images and can thus adapt to person-specific facial dynamics. We also propose data augmentation and regularization schemes that are necessary to prevent overfitting and support generalizability of the learned representations. We evaluated our approach in a randomized user study. The results indicate superior performance compared to the state-of-the-art in terms of motion transfer quality and temporal consistency.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
