TextToon：基于单视频的实时文本卡通化头部头像生成
TextToon: Real-Time Text Toonify Head Avatar from Single Video
AI总结：
本研究提出TextToon方法，基于单目视频和文本指令生成可实时驱动的高保真卡通化头像，通过条件嵌入三平面、自适应像素平移网络等优化3D高斯溅射的风格化效果，实现多设备实时运行，性能优于现有方法。
摘要：
我们提出了TextToon，一种可生成可驱动的卡通化头像的方法。给定一段短的单目视频序列以及一条关于头像风格的文字指令，我们的模型能够生成高保真的卡通化头像，该头像可由任意身份的另一视频实时驱动。现有相关工作严重依赖多视图建模，通过纹理嵌入以静态方式恢复几何结构，导致控制受限。多视图视频输入也使得这些模型难以在实际应用中部署。为解决这些问题，我们采用条件嵌入Tri-plane（三平面）在高斯变形场中学习逼真且风格化的面部表征。此外，我们通过引入自适应像素平移神经网络并利用patch-aware（块感知）对比学习，拓展了3D Gaussian Splatting（三维高斯溅射）的风格化能力，以生成高质量图像。为推动我们的工作面向消费级应用，我们开发了一套实时系统，在GPU设备上运行速度可达48 FPS，在移动设备上为15-18 FPS。大量实验表明，我们的方法在生成文本驱动头像的质量和实时动画效果方面均优于现有方法。更多细节请参阅我们的项目页面：https://songluchuan.github.io/TextToon/。
英文摘要：
We propose TextToon, a method to generate a drivable toonified avatar. Given a short monocular video sequence and a written instruction about the avatar style, our model can generate a high-fidelity toonified avatar that can be driven in real-time by another video with arbitrary identities. Existing related works heavily rely on multi-view modeling to recover geometry via texture embeddings, presented in a static manner, leading to control limitations. The multi-view video input also makes it difficult to deploy these models in real-world applications. To address these issues, we adopt a conditional embedding Tri-plane to learn realistic and stylized facial representations in a Gaussian deformation field. Additionally, we expand the stylization capabilities of 3D Gaussian Splatting by introducing an adaptive pixel-translation neural network and leveraging patch-aware contrastive learning to achieve high-quality images. To push our work into consumer applications, we develop a real-time system that can operate at 48 FPS on a GPU machine and 15-18 FPS on a mobile machine. Extensive experiments demonstrate the efficacy of our approach in generating textual avatars over existing methods in terms of quality and real-time animation. Please refer to our project page for more details: https://songluchuan.github.io/TextToon/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
