用于网格无关面部表情克隆的神经蒙皮
Neural Face Skinning for Mesh-agnostic Facial Expression Cloning
AI总结：
提出一种结合全局与局部变形模型的神经蒙皮方法，通过预测顶点蒙皮权重局部化全局潜在代码，实现网格无关的高保真面部表情克隆与直观编辑。
摘要：
准确地将面部表情重定向到面部网格并支持操控，是面部动画重定向中的关键挑战。近期的深度学习方法通过将面部表情编码为全局潜在代码来解决此问题，但它们往往无法捕捉局部区域的细粒度细节。尽管一些方法通过局部传递变形来提高局部精度，但这通常会使面部表情的整体控制变得复杂。为了解决这一问题，我们提出了一种结合全局和局部变形模型优势的方法。我们的方法能够实现直观的控制和跨不同面部网格的详细表情克隆，而不受其底层结构的影响。核心思想是将全局潜在代码对目标网格的影响局部化。我们的模型通过来自预定义分割标签的间接监督，学习预测目标面部网格每个顶点的蒙皮权重。这些预测的权重将全局潜在代码局部化，即使对于具有未见形状的网格，也能实现精确且特定于区域的变形。我们使用基于面部动作编码系统（FACS）的混合形状来监督潜在代码，以确保可解释性并允许对生成的动画进行直接编辑。通过大量实验，我们证明了在表情保真度、变形传递精度以及跨不同网格结构的适应性方面，我们的方法优于现有最先进的方法。
英文摘要：
Accurately retargeting facial expressions to a face mesh while enabling manipulation is a key challenge in facial animation retargeting. Recent deep-learning methods address this by encoding facial expressions into a global latent code, but they often fail to capture fine-grained details in local regions. While some methods improve local accuracy by transferring deformations locally, this often complicates overall control of the facial expression. To address this, we propose a method that combines the strengths of both global and local deformation models. Our approach enables intuitive control and detailed expression cloning across diverse face meshes, regardless of their underlying structures. The core idea is to localize the influence of the global latent code on the target mesh. Our model learns to predict skinning weights for each vertex of the target face mesh through indirect supervision from predefined segmentation labels. These predicted weights localize the global latent code, enabling precise and region-specific deformations even for meshes with unseen shapes. We supervise the latent code using Facial Action Coding System (FACS)-based blendshapes to ensure interpretability and allow straightforward editing of the generated animation. Through extensive experiments, we demonstrate improved performance over state-of-the-art methods in terms of expression fidelity, deformation transfer accuracy, and adaptability across diverse mesh structures.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
