无需注册的可学习多视角面部密集语义对应捕捉
Registration-Free Learnable Multi-View Capture of Faces in Dense Semantic Correspondence
AI总结：
本文提出MOCHI框架，无需注册数据直接从多视角图像预测3D面部，通过伪线性逆运动学求解器确保拓扑一致性，改进损失函数提升重建精度。
摘要：
最近的框架如ToFu和TEMPEH通过从校准多视角图像直接预测3D网格，为传统注册流程提供了自动化替代方案。然而，这些学习方法在训练监督中依赖于缓慢的手动注册流程。我们通过MOCHI（多视角优化对应头部图像）克服了这一限制，该框架在无需注册训练数据的情况下训练。MOCHI通过伪线性逆运动学求解器消除注册数据依赖，利用从合成数据训练的2D关键点预测器引导语义对齐。我们的分析进一步表明，标准点到表面距离在无注册设置中导致训练不稳定和视觉伪影。我们提出基于点图和法线的损失函数，提供更平滑的梯度和更优的重建保真度。最后，我们引入了测试时优化方案，在几十次迭代中优化网络权重。这种方法在效率和精度之间架起桥梁，使MOCHI在重建准确性和视觉质量上优于传统劳动密集型流程。代码和模型在：https://filby89.github.io/mochi公开。
英文摘要：
Recent frameworks like ToFu and TEMPEH provide an automated alternative to classical registration pipelines by predicting 3D meshes in dense semantic correspondence directly from calibrated multi-view images. However, these learning-based methods rely on the slow, manual registration pipelines they aim to replace for their training supervision. We overcome this limitation with MOCHI (Multi-view Optimizable Correspondence of Heads from Images), a multi-view 3D face prediction framework trained without requiring registered training data. MOCHI eliminates the registration data dependency by enforcing topological consistency through a pseudo-linear inverse kinematic solver. Semantic alignment is guided by dense keypoints from a 2D landmark predictor trained exclusively on synthetic data. Our analysis further reveals that standard point-to-surface distances induce training instabilities and visual artifacts in registration-free settings. We propose pointmap- and normal-based losses instead, which provide smoother gradients and superior reconstruction fidelity. Finally, we introduce a test-time optimization scheme that refines network weights over a few dozen iterations. This approach bridges the gap between feed-forward efficiency and iterative optimization precision, allowing MOCHI to outperform traditional labor-intensive pipelines in both reconstruction accuracy and visual quality. Code and model are public at: https://filby89.github.io/mochi.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
