神经面部绑定:野外面部网格的动画与重定向
Neural Face Rigging for Animating and Retargeting Facial Meshes in the Wild
AI总结:
提出神经面部绑定(NFR),一种端到端深度学习方法,无需手动混合形状即可自动绑定和重定向野外任意面部网格,提供可解释的编辑参数并生成细粒度细节。
摘要:
我们提出了一种端到端的深度学习方法,用于自动绑定和重定向野外人类面部3D模型。我们的方法称为神经面部绑定(NFR),具有三个关键特性:(i)NFR的表情空间保持人类可解释的编辑参数,以便进行艺术控制;(ii)NFR可轻松应用于具有不同连接性和表情的任意面部网格;(iii)NFR能够编码并生成任意受试者执行的复杂表情的细粒度细节。据我们所知,NFR是首个无需手动创建混合形状或对应关系即可为野外面部网格提供逼真且可控变形的方法。我们设计了一个变形自编码器,并通过多数据集训练方案进行训练,该方案受益于两个数据源的独特优势:一个具有FACS中可解释控制参数的线性3DMM,以及具有细粒度细节的真实面部4D捕捉。通过各种实验,我们展示了NFR能够自动生成逼真且准确的面部变形,适用于广泛现有数据集以及野外的噪声面部扫描,同时提供艺术家可控、可编辑的参数。
英文摘要:
We propose an end-to-end deep-learning approach for automatic rigging and retargeting of 3D models of human faces in the wild. Our approach, called Neural Face Rigging (NFR), holds three key properties: (i) NFR's expression space maintains human-interpretable editing parameters for artistic controls; (ii) NFR is readily applicable to arbitrary facial meshes with different connectivity and expressions; (iii) NFR can encode and produce fine-grained details of complex expressions performed by arbitrary subjects. To the best of our knowledge, NFR is the first approach to provide realistic and controllable deformations of in-the-wild facial meshes, without the manual creation of blendshapes or correspondence. We design a deformation autoencoder and train it through a multi-dataset training scheme, which benefits from the unique advantages of two data sources: a linear 3DMM with interpretable control parameters as in FACS, and 4D captures of real faces with fine-grained details. Through various experiments, we show NFR's ability to automatically produce realistic and accurate facial deformations across a wide range of existing datasets as well as noisy facial scans in-the-wild, while providing artist-controlled, editable parameters.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。