面向单目动态人体建模的运动导向组合神经辐射场
Motion-Oriented Compositional Neural Radiance Fields for Monocular Dynamic Human Modeling
AI总结：
本文提出MoCo-NeRF，通过将非刚性运动建模为辐射残差场并利用刚性辐射场先验，实现单目人体视频的高质量自由视点渲染，支持多主体同时训练，性能领先。
摘要：
本文介绍了运动导向组合神经辐射场（MoCo-NeRF），一种通过新颖的非刚性运动建模方法对单目人体视频进行自由视点渲染的框架。在动态穿衣人体的背景下，复杂的布料动力学产生非刚性运动，这些运动本质上不同于骨骼关节运动，且对渲染质量至关重要。传统方法将非刚性运动建模为骨骼变换之外的空间（3D）偏差。然而，由于缺乏直接监督导致的学习复杂度高，这种方法要么耗时，要么难以达到最优质量。为解决此问题，我们提出一种新方法，将非刚性运动建模为辐射残差场，以利用渲染中更直接的色彩监督，并将刚性辐射场作为先验以减少学习过程的复杂度。我们的方法利用单一多分辨率哈希编码（MHE）同时从刚性骨骼运动中学习规范T姿态表示，以及用于非刚性运动的辐射残差场。此外，为进一步提高训练效率和可用性，得益于我们有效的非刚性运动建模设计，我们将MoCo-NeRF扩展以支持在单一框架内同时训练多个主体。这种可扩展性通过集成全局MHE、可学习身份编码以及多个局部MHE实现。我们在ZJU-MoCap和MonoCap上展示了广泛的结果，明确证明了在单主体和多主体设置中的最先进性能。代码和模型将在项目页面公开：https://stevejaehyeok.github.io/publications/moco-nerf。
英文摘要：
This paper introduces Motion-oriented Compositional Neural Radiance Fields (MoCo-NeRF), a framework designed to perform free-viewpoint rendering of monocular human videos via novel non-rigid motion modeling approach. In the context of dynamic clothed humans, complex cloth dynamics generate non-rigid motions that are intrinsically distinct from skeletal articulations and critically important for the rendering quality. The conventional approach models non-rigid motions as spatial (3D) deviations in addition to skeletal transformations. However, it is either time-consuming or challenging to achieve optimal quality due to its high learning complexity without a direct supervision. To target this problem, we propose a novel approach of modeling non-rigid motions as radiance residual fields to benefit from more direct color supervision in the rendering and utilize the rigid radiance fields as a prior to reduce the complexity of the learning process. Our approach utilizes a single multiresolution hash encoding (MHE) to concurrently learn the canonical T-pose representation from rigid skeletal motions and the radiance residual field for non-rigid motions. Additionally, to further improve both training efficiency and usability, we extend MoCo-NeRF to support simultaneous training of multiple subjects within a single framework, thanks to our effective design for modeling non-rigid motions. This scalability is achieved through the integration of a global MHE and learnable identity codes in addition to multiple local MHEs. We present extensive results on ZJU-MoCap and MonoCap, clearly demonstrating state-of-the-art performance in both single- and multi-subject settings. The code and model will be made publicly available at the project page: https://stevejaehyeok.github.io/publications/moco-nerf.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
