arXiv 2411.030862024-11-05cs.CVcs.AI

HFGaussian：学习具有集成人体特征的可泛化高斯人体

HFGaussian: Learning Generalizable Gaussian Human with Integrated Human Features

I3S-CNRS/Université Côte d’Azur（蔚蓝海岸大学I3S-CNRS研究所）；University of Technology Sydney（悉尼科技大学）；Brown University（布朗大学）

Arnab Dey, Cheng-You Lu, Andrew I. Comport, Srinath Sridhar, Chin-Teng Lin, Jean Martinet

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

提出HFGaussian，一种基于可泛化高斯溅射的新方法，从稀疏图像实时估计新视角及3D骨架、关键点、密集姿态等人体特征，在25 FPS下实现最先进性能。

摘要：

辐射场渲染的最新进展在3D场景表示方面显示出有前景的结果，其中基于高斯溅射的技术因其质量和效率而成为最先进的方法。高斯溅射被广泛用于各种应用，包括3D人体表示。然而，先前的3D高斯溅射方法要么使用参数化身体模型作为附加信息，要么未能提供任何底层结构，如人体生物力学特征，而这些特征对于不同应用至关重要。在本文中，我们提出了一种名为HFGaussian的新方法，该方法可以从稀疏输入图像中实时（25 FPS）估计新视角和人体特征，如3D骨架、3D关键点和密集姿态。所提出的方法利用可泛化的高斯溅射技术来表示人体对象及其相关特征，从而实现高效且可泛化的重建。通过结合姿态回归网络和特征溅射技术与高斯溅射，HFGaussian展示了相对于现有3D人体方法的改进能力，展示了具有集成生物力学的3D人体表示的潜力。我们针对最新的人体高斯溅射和姿态估计最先进技术对HFGaussian方法进行了全面评估，证明了其实时、最先进的性能。

英文摘要：

Recent advancements in radiance field rendering show promising results in 3D scene representation, where Gaussian splatting-based techniques emerge as state-of-the-art due to their quality and efficiency. Gaussian splatting is widely used for various applications, including 3D human representation. However, previous 3D Gaussian splatting methods either use parametric body models as additional information or fail to provide any underlying structure, like human biomechanical features, which are essential for different applications. In this paper, we present a novel approach called HFGaussian that can estimate novel views and human features, such as the 3D skeleton, 3D key points, and dense pose, from sparse input images in real time at 25 FPS. The proposed method leverages generalizable Gaussian splatting technique to represent the human subject and its associated features, enabling efficient and generalizable reconstruction. By incorporating a pose regression network and the feature splatting technique with Gaussian splatting, HFGaussian demonstrates improved capabilities over existing 3D human methods, showcasing the potential of 3D human representations with integrated biomechanics. We thoroughly evaluate our HFGaussian method against the latest state-of-the-art techniques in human Gaussian splatting and pose estimation, demonstrating its real-time, state-of-the-art performance.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。