可重照明和动态高斯人偶重建从单目视频
Relightable and Dynamic Gaussian Avatar Reconstruction from Monocular Video
AI总结：
本文提出了一种基于3DGS的人偶建模框架RnD-Avatar，通过动态皮肤权重和新的正则化方法实现高保真几何细节，支持在任意光照条件下进行逼真渲染和重照明。
摘要：
从单目视频建模可重照明和可动画的人偶是一项长期且具有挑战性的任务。最近，神经辐射场（NeRF）和3D高斯点撒（3DGS）方法已被用于重建人偶。然而，由于与身体运动相关的几何细节不足，如衣物褶皱，它们往往产生不满意的逼真结果。在本文中，我们提出了一种基于3DGS的人偶建模框架，称为可重照明和动态高斯人偶（RnD-Avatar），该框架能够提供准确的姿势变异变形以获得高保真的几何细节。为此，我们引入了动态皮肤权重，这些权重基于姿态定义了人偶的可动性，同时学习由身体运动引起的额外变形。我们还引入了一种新的正则化方法，在稀疏视觉线索下捕捉精细的几何细节。此外，我们提出了一个新的多视角数据集，具有变化的光照条件，以评估重照明。我们的框架能够在新的姿态和视角下实现逼真的渲染，同时支持在任意光照条件下实现逼真的光照效果。我们的方法在新的视角合成、新的姿态渲染和重照明方面达到了最先进的性能。
英文摘要：
Modeling relightable and animatable human avatars from monocular video is a long-standing and challenging task. Recently, Neural Radiance Field (NeRF) and 3D Gaussian Splatting (3DGS) methods have been employed to reconstruct the avatars. However, they often produce unsatisfactory photo-realistic results because of insufficient geometrical details related to body motion, such as clothing wrinkles. In this paper, we propose a 3DGS-based human avatar modeling framework, termed as Relightable and Dynamic Gaussian Avatar (RnD-Avatar), that presents accurate pose-variant deformation for high-fidelity geometrical details. To achieve this, we introduce dynamic skinning weights that define the human avatar's articulation based on pose while also learning additional deformations induced by body motion. We also introduce a novel regularization to capture fine geometric details under sparse visual cues. Furthermore, we present a new multi-view dataset with varied lighting conditions to evaluate relight. Our framework enables realistic rendering of novel poses and views while supporting photo-realistic lighting effects under arbitrary lighting conditions. Our method achieves state-of-the-art performance in novel view synthesis, novel pose rendering, and relighting.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
