用于动态场景重建的Motion Blender Gaussian Splatting
Motion Blender Gaussian Splatting for Dynamic Scene Reconstruction
AI总结：
针对现有动态场景高斯溅射重建方法运动可控性不足的问题，提出采用运动图显式表示、对偶四元数蒙皮传播运动的MBGS框架，实现了SOTA重建效果与多场景应用潜力。
摘要：
高斯溅射已成为高保真动态场景重建的有力工具。然而，现有方法主要依赖隐式运动表示，例如将运动编码到神经网络或逐高斯参数中，这使得进一步操控重建后的运动十分困难。这种显式可控性的缺失导致现有方法仅能回放已记录的运动，阻碍了其在机器人领域的更广泛应用。为解决这一问题，我们提出了Motion Blender Gaussian Splatting（MBGS，运动混合器高斯溅射），这是一种采用运动图作为显式且稀疏运动表示的新型框架。图中连杆的运动通过对偶四元数蒙皮传播至各个高斯，由可学习的权重绘制函数决定每个连杆的影响范围。运动图与3D高斯通过可微渲染从输入视频中联合优化。实验表明，MBGS在极具挑战性的iPhone数据集上达到了当前最优性能，在HyperNeRF数据集上也具备竞争力。我们展示了该方法在生成新颖物体姿态动画、合成真实机器人演示以及通过视觉规划预测机器人动作方面的应用潜力。源代码、模型与视频演示可访问http://mlzxy.github.io/motion-blender-gs获取。
英文摘要：
Gaussian splatting has emerged as a powerful tool for high-fidelity reconstruction of dynamic scenes. However, existing methods primarily rely on implicit motion representations, such as encoding motions into neural networks or per-Gaussian parameters, which makes it difficult to further manipulate the reconstructed motions. This lack of explicit controllability limits existing methods to replaying recorded motions only, which hinders a wider application in robotics. To address this, we propose Motion Blender Gaussian Splatting (MBGS), a novel framework that uses motion graphs as an explicit and sparse motion representation. The motion of a graph's links is propagated to individual Gaussians via dual quaternion skinning, with learnable weight painting functions that determine the influence of each link. The motion graphs and 3D Gaussians are jointly optimized from input videos via differentiable rendering. Experiments show that MBGS achieves state-of-the-art performance on the highly challenging iPhone dataset while being competitive on HyperNeRF. We demonstrate the application potential of our method in animating novel object poses, synthesizing real robot demonstrations, and predicting robot actions through visual planning. The source code, models, video demonstrations can be found at http://mlzxy.github.io/motion-blender-gs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
