MaGS：利用网格吸附高斯溅射重建与模拟动态三维物体
MaGS: Reconstructing and Simulating Dynamic 3D Objects with Mesh-adsorbed Gaussian Splatting
AI总结：
本文提出MaGS方法，通过将3D高斯约束在网格附近形成相互吸附的表示，结合RMD-Net、RGD-Net和MPE-Net三个网络，实现动态三维物体的高质量重建与模拟，并在多个数据集上取得最先进性能。
摘要：
三维重建与模拟虽然相互关联，但目标不同：重建需要灵活的3D表示以适应多样场景，而模拟则需要结构化表示以有效建模运动原理。本文提出网格吸附高斯溅射（MaGS）方法来解决这一挑战。MaGS将3D高斯约束在网格附近漫游，形成一种网格与高斯相互吸附的3D表示。这种表示同时利用了3D高斯的渲染灵活性和网格的结构化特性。为实现这一目标，我们引入了RMD-Net，该网络从视频数据中学习运动先验以细化网格变形，同时引入RGD-Net，建模网格与高斯之间的相对位移，以在网格约束下提升渲染保真度。为泛化到输入视频之外的新颖、用户自定义变形，且不依赖时序数据，我们提出了MPE-Net，利用网格固有信息来引导RMD-Net和RGD-Net。由于网格的通用性，MaGS兼容多种变形先验，如ARAP、SMPL和软体物理模拟。在D-NeRF、DG-Mesh和PeopleSnapshot数据集上的大量实验表明，MaGS在重建和模拟两方面均达到了最先进的性能。
英文摘要：
3D reconstruction and simulation, although interrelated, have distinct objectives: reconstruction requires a flexible 3D representation that can adapt to diverse scenes, while simulation needs a structured representation to model motion principles effectively. This paper introduces the Mesh-adsorbed Gaussian Splatting (MaGS) method to address this challenge. MaGS constrains 3D Gaussians to roam near the mesh, creating a mutually adsorbed mesh-Gaussian 3D representation. Such representation harnesses both the rendering flexibility of 3D Gaussians and the structured property of meshes. To achieve this, we introduce RMD-Net, a network that learns motion priors from video data to refine mesh deformations, alongside RGD-Net, which models the relative displacement between the mesh and Gaussians to enhance rendering fidelity under mesh constraints. To generalize to novel, user-defined deformations beyond input video without reliance on temporal data, we propose MPE-Net, which leverages inherent mesh information to bootstrap RMD-Net and RGD-Net. Due to the universality of meshes, MaGS is compatible with various deformation priors such as ARAP, SMPL, and soft physics simulation. Extensive experiments on the D-NeRF, DG-Mesh, and PeopleSnapshot datasets demonstrate that MaGS achieves state-of-the-art performance in both reconstruction and simulation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
