FMGS-Avatar：基于基础模型先验的网格引导二维高斯泼溅用于三维单目虚拟化身重建
FMGS-Avatar: Mesh-Guided 2D Gaussian Splatting with Foundation Model Priors for 3D Monocular Avatar Reconstruction
AI总结：
FMGS-Avatar通过网格引导的二维高斯泼溅约束原语附着于模板网格以保持表面细节，并利用基础模型先验结合选择性梯度隔离的协调训练策略，显著提升单目视频三维人类化身重建的几何精度与外观保真度。
摘要：
从单目视频重建高保真可动画人类化身仍然具有挑战性，因为单视角观测中的几何信息不足。尽管最近的三维高斯泼溅方法显示出潜力，但由于三维高斯原语的自由形式特性，它们在保持表面细节方面存在困难。为了解决表示局限性和信息稀缺问题，我们提出了一种新方法FMGS-Avatar，它整合了两个关键创新。首先，我们引入了网格引导的二维高斯泼溅，其中二维高斯原语直接附着在模板网格面上，并约束其位置、旋转和运动，从而实现优越的表面对齐和几何细节保持。其次，我们利用在大规模数据集上训练的基础模型（如Sapiens）来补充单目视频中有限的视觉线索。然而，当从基础模型中蒸馏多模态先验知识时，由于不同模态表现出不同的参数敏感性，可能会出现相互冲突的优化目标。我们通过一种具有选择性梯度隔离的协调训练策略来解决这一问题，使每个损失分量能够优化其相关参数而不受干扰。通过增强表示和协调信息蒸馏的结合，我们的方法显著推进了三维单目人类化身重建。实验评估表明，与现有方法相比，我们的方法具有优越的重建质量，在几何精度和外观保真度方面取得了显著提升，同时提供了丰富的语义信息。此外，在共享规范空间内蒸馏的先验知识自然地实现了在新视角和新姿态下空间和时间一致的渲染。
英文摘要：
Reconstructing high-fidelity animatable human avatars from monocular videos remains challenging due to insufficient geometric information in single-view observations. While recent 3D Gaussian Splatting methods have shown promise, they struggle with surface detail preservation due to the free-form nature of 3D Gaussian primitives. To address both the representation limitations and information scarcity, we propose a novel method, \textbf{FMGS-Avatar}, that integrates two key innovations. First, we introduce Mesh-Guided 2D Gaussian Splatting, where 2D Gaussian primitives are attached directly to template mesh faces with constrained position, rotation, and movement, enabling superior surface alignment and geometric detail preservation. Second, we leverage foundation models trained on large-scale datasets, such as Sapiens, to complement the limited visual cues from monocular videos. However, when distilling multi-modal prior knowledge from foundation models, conflicting optimization objectives can emerge as different modalities exhibit distinct parameter sensitivities. We address this through a coordinated training strategy with selective gradient isolation, enabling each loss component to optimize its relevant parameters without interference. Through this combination of enhanced representation and coordinated information distillation, our approach significantly advances 3D monocular human avatar reconstruction. Experimental evaluation demonstrates superior reconstruction quality compared to existing methods, with notable gains in geometric accuracy and appearance fidelity while providing rich semantic information. Additionally, the distilled prior knowledge within a shared canonical space naturally enables spatially and temporally consistent rendering under novel views and poses.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
