arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2403.11427cs.CV

BAGS:利用扩散先验从单目视频构建可动画化的高斯泼溅

BAGS: Building Animatable Gaussian Splatting from a Monocular Video with Diffusion Priors

National Key Lab of General AI, Peking University(通用人工智能全国重点实验室);Shandong University(山东大学);The Hong Kong University of Science and Technology(香港科技大学)

Tingyang Zhang, Qingzhe Gao, Weiyu Li, Libin Liu, Baoquan Chen

AI总结:

本文提出BAGS方法,利用扩散先验从单目视频构建可动画化三维高斯泼溅,加速训练渲染并支持有限视角,通过刚性正则化提升性能,优于现有方法。

摘要:

可动画化的三维重建在多个领域具有重要应用,目前主要依赖艺术家的手工创作。近期,一些研究成功地从单目视频构建了可动画化的三维模型。然而,这些方法要求输入视频中物体具有足够的视角覆盖,并且通常需要大量的时间和计算成本进行训练和渲染。这一局限性限制了其实际应用。在本工作中,我们提出了一种利用扩散先验从单目视频构建可动画化三维高斯泼溅的方法。三维高斯表示显著加速了训练和渲染过程,而扩散先验使得该方法能够在有限视角下学习三维模型。我们还提出了刚性正则化以增强对先验的利用。我们在各种真实世界视频上进行了广泛评估,展示了其相对于当前最先进方法的优越性能。

英文摘要:

Animatable 3D reconstruction has significant applications across various fields, primarily relying on artists' handcraft creation. Recently, some studies have successfully constructed animatable 3D models from monocular videos. However, these approaches require sufficient view coverage of the object within the input video and typically necessitate significant time and computational costs for training and rendering. This limitation restricts the practical applications. In this work, we propose a method to build animatable 3D Gaussian Splatting from monocular video with diffusion priors. The 3D Gaussian representations significantly accelerate the training and rendering process, and the diffusion priors allow the method to learn 3D models with limited viewpoints. We also present the rigid regularization to enhance the utilization of the priors. We perform an extensive evaluation across various real-world videos, demonstrating its superior performance compared to the current state-of-the-art methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑