TalkinNeRF：用于全身说话人类的可动画化神经场
TalkinNeRF: Animatable Neural Fields for Full-Body Talking Humans
AI总结：
提出TalkinNeRF框架，从单目视频学习全身说话人类的动态神经辐射场，统一表示身体、面部和手部运动，支持多身份训练与未见姿态下的稳健动画。
摘要：
我们提出了一种新颖的框架，能够从单目视频中学习全身说话人类的动态神经辐射场（NeRF）。先前的工作仅表示身体姿态或面部。然而，人类通过全身进行交流，结合了身体姿态、手势以及面部表情。在本工作中，我们提出了TalkinNeRF，一种统一的基于NeRF的网络，用于表示整体的4D人体运动。给定一个对象的单目视频，我们分别学习身体、面部和手部的对应模块，并将它们组合在一起以生成最终结果。为了捕捉复杂的手指关节运动，我们为手部学习了一个额外的变形场。我们的多身份表示实现了对多个对象的同时训练，以及在完全未见过的姿态下的稳健动画。在仅给定短视频作为输入时，它也能泛化到新身份。我们展示了在制作全身说话人类动画方面的最先进性能，具有细粒度的手部关节运动和面部表情。
英文摘要：
We introduce a novel framework that learns a dynamic neural radiance field (NeRF) for full-body talking humans from monocular videos. Prior work represents only the body pose or the face. However, humans communicate with their full body, combining body pose, hand gestures, as well as facial expressions. In this work, we propose TalkinNeRF, a unified NeRF-based network that represents the holistic 4D human motion. Given a monocular video of a subject, we learn corresponding modules for the body, face, and hands, that are combined together to generate the final result. To capture complex finger articulation, we learn an additional deformation field for the hands. Our multi-identity representation enables simultaneous training for multiple subjects, as well as robust animation under completely unseen poses. It can also generalize to novel identities, given only a short video as input. We demonstrate state-of-the-art performance for animating full-body talking humans, with fine-grained hand articulation and facial expressions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
