基于循环 StyleGAN 生成器的语言引导人脸动画
Language-Guided Face Animation by Recurrent StyleGAN-based Generator
AI总结：
本文提出语言引导人脸动画任务，通过循环运动生成器结合预训练 StyleGAN，并设计身份保持、运动平滑和对比损失，实现从单张静态图像生成高质量视频。
摘要：
近年来关于语言引导图像操纵的研究展示了语言在提供丰富语义方面的强大能力，尤其是对于人脸图像。然而，语言中的另一类自然信息——运动，较少被探索。在本文中，我们利用运动信息并研究了一项新任务，即语言引导人脸动画，旨在借助语言使静态人脸图像动起来。为了更好地利用语言中的语义和运动信息，我们提出了一个简单而有效的框架。具体来说，我们提出了一个循环运动生成器，从语言中提取一系列语义和运动信息，并将其与视觉信息一起输入预训练的 StyleGAN，以生成高质量帧。为了优化所提出的框架，我们设计了三个精心构造的损失函数，包括保持人脸身份的正则化损失、确保运动平滑的路径长度正则化损失，以及使单一模型能够在各种语言引导下进行视频合成的对比损失。在多样化领域（例如，真人脸、动漫脸和狗脸）上进行的广泛实验，包括定性和定量评估，证明了我们的模型在语言引导下从单张静态图像生成高质量、逼真视频方面的优越性。代码将发布于 https://github.com/TiankaiHang/language-guided-animation.git。
英文摘要：
Recent works on language-guided image manipulation have shown great power of language in providing rich semantics, especially for face images. However, the other natural information, motions, in language is less explored. In this paper, we leverage the motion information and study a novel task, language-guided face animation, that aims to animate a static face image with the help of languages. To better utilize both semantics and motions from languages, we propose a simple yet effective framework. Specifically, we propose a recurrent motion generator to extract a series of semantic and motion information from the language and feed it along with visual information to a pre-trained StyleGAN to generate high-quality frames. To optimize the proposed framework, three carefully designed loss functions are proposed including a regularization loss to keep the face identity, a path length regularization loss to ensure motion smoothness, and a contrastive loss to enable video synthesis with various language guidance in one single model. Extensive experiments with both qualitative and quantitative evaluations on diverse domains (\textit{e.g.,} human face, anime face, and dog face) demonstrate the superiority of our model in generating high-quality and realistic videos from one still image with the guidance of language. Code will be available at https://github.com/TiankaiHang/language-guided-animation.git.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
