MeGA: 混合网格-高斯头像用于高保真渲染和头像编辑
MeGA: Hybrid Mesh-Gaussian Head Avatar for High-Fidelity Rendering and Head Editing
AI总结：
MeGA通过混合网格和高斯表示，实现高保真头像渲染与编辑，提升面部和头发的细节与真实感。
摘要：
从多视角视频创建高保真的头像对于许多AR/VR应用来说是一个核心问题。然而，现有方法通常难以同时获得高质量的渲染结果，因为它们使用单一的表示来建模具有截然不同特征的组件（例如，皮肤与头发）。在本文中，我们提出了混合网格-高斯头像（MeGA），通过更合适的表示来建模不同的头部组件。具体来说，我们选择增强的FLAME网格作为面部表示，并预测一个UV位移图，以提供顶点偏移，以改善个性化的几何细节。为了实现逼真的渲染，我们使用延迟神经渲染获取面部颜色，并将神经纹理分解为三个有意义的部分。对于头发建模，我们首先使用3D高斯点散布构建一个静态的规范头发。随后，进一步应用刚体变换和基于MLP的变形场来处理复杂的动态表情。结合我们的遮挡意识混合，MeGA为整个头部生成更高保真的渲染，并自然支持更多下游任务。在NeRSemble数据集上的实验表明，我们的设计是有效的，优于先前的最先进方法，并支持各种编辑功能，包括发型改变和纹理编辑。
英文摘要：
Creating high-fidelity head avatars from multi-view videos is a core issue for many AR/VR applications. However, existing methods usually struggle to obtain high-quality renderings for all different head components simultaneously since they use one single representation to model components with drastically different characteristics (e.g., skin vs. hair). In this paper, we propose a Hybrid Mesh-Gaussian Head Avatar (MeGA) that models different head components with more suitable representations. Specifically, we select an enhanced FLAME mesh as our facial representation and predict a UV displacement map to provide per-vertex offsets for improved personalized geometric details. To achieve photorealistic renderings, we obtain facial colors using deferred neural rendering and disentangle neural textures into three meaningful parts. For hair modeling, we first build a static canonical hair using 3D Gaussian Splatting. A rigid transformation and an MLP-based deformation field are further applied to handle complex dynamic expressions. Combined with our occlusion-aware blending, MeGA generates higher-fidelity renderings for the whole head and naturally supports more downstream tasks. Experiments on the NeRSemble dataset demonstrate the effectiveness of our designs, outperforming previous state-of-the-art methods and supporting various editing functionalities, including hairstyle alteration and texture editing.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
