通过可控局部变形场实现隐式神经头部合成
Implicit Neural Head Synthesis via Controllable Local Deformation Fields
浏览论文内容
中文总结 AI 辅助
针对单目视频中3D头部化身的精细局部控制问题，提出基于3DMM参数和关键点的多个局部隐式变形场，并以稀疏性约束实现更锐利的口腔、非对称表情及面部细节合成。
中文摘要 AI 辅助
从2D视频高质量重建可控3D头部化身，对于电影、游戏和远程呈现中的虚拟人应用极具价值。神经隐式场提供了一种强大的表示，可建模具有个性化形状、表情以及头发和口腔内部等面部部件的3D头部化身，超越了线性3D可变形模型（3DMM）。然而，现有方法无法建模具有精细面部特征的人脸，也无法对从单目视频外推非对称表情的面部部件进行局部控制。此外，大多数方法仅以局部性较差的3DMM参数为条件，并通过全局神经场解析局部特征。本文基于将全局变形场分解为局部变形场的部件级隐式形状模型。新公式通过基于3DMM的参数和代表性面部关键点，建模多个具有局部语义绑定控制的隐式变形场。此外，本文提出局部控制损失和注意力掩码机制，以促进每个学习到的变形场的稀疏性。与以往的隐式单目方法相比，该公式渲染出更锐利、局部可控的非线性变形，尤其是口腔内部、非对称表情和面部细节。
英文摘要
High-quality reconstruction of controllable 3D head avatars from 2D videos is highly desirable for virtual human applications in movies, games, and telepresence. Neural implicit fields provide a powerful representation to model 3D head avatars with personalized shape, expressions, and facial parts, e.g., hair and mouth interior, that go beyond the linear 3D morphable model (3DMM). However, existing methods do not model faces with fine-scale facial features, or local control of facial parts that extrapolate asymmetric expressions from monocular videos. Further, most condition only on 3DMM parameters with poor(er) locality, and resolve local features with a global neural field. We build on part-based implicit shape models that decompose a global deformation field into local ones. Our novel formulation models multiple implicit deformation fields with local semantic rig-like control via 3DMM-based parameters, and representative facial landmarks. Further, we propose a local control loss and attention mask mechanism that promote sparsity of each learned deformation field. Our formulation renders sharper locally controllable nonlinear deformations than previous implicit monocular approaches, especially mouth interior, asymmetric expressions, and facial details.
发表机构
- Carnegie Mellon University（卡内基梅隆大学）
- Flawless AI
机构由 AI 辅助整理，请以论文原文为准。
