ManiSplat: 基于解耦3D高斯泼溅的单目视频操作轨迹合成
ManiSplat: Manipulation Trajectory Synthesis from Monocular Video via Decoupled 3D Gaussian Splatting
AI总结：
提出ManiSplat框架，通过图结构解耦表示和任务导向时空对齐，从单目视频重建可控的3D高斯数字孪生，支持机器人操作任务与策略学习。
摘要：
从真实世界观测中重建动态且可交互的3D场景仍然是计算机视觉和机器人学中的一个基本挑战。尽管3D高斯泼溅的最新进展实现了高保真静态重建，但由于复杂的接触交互和突变的姿态变化，将其扩展到具有关节机器人和可操作物体的交互环境仍然困难。为了解决这些挑战，我们引入了ManiSplat，一个统一的框架，直接从单目自我中心机器人视频重建可控且解耦的高斯数字孪生。我们的方法引入了一种图结构解耦表示，将机器人、物体和背景分离为独立可优化的高斯子场，并组织在场景图中。为了确保稳定性，我们提出了一个任务导向的时空对齐模块，利用操作任务的内在逻辑——在运动和技能阶段之间交替——来构建准确的伪真实轨迹。最后，联合光度-几何优化确保重建场景在时间上连贯、物理上一致且可用于仿真。大量实验表明，我们的方法以高保真度和可控性重建了交互驱动的动态场景，有效支持下游机器人任务和策略学习。
英文摘要：
Reconstructing dynamic and interactive 3D scenes from real-world observations remains a fundamental challenge in computer vision and robotics. While recent advances in 3D Gaussian Splatting have enabled high-fidelity static reconstruction, extending it to interactive environments with articulated robots and manipulable objects remains difficult due to complex contact interactions and abrupt pose changes. To address these challenges, we introduce ManiSplat, a unified framework that reconstructs controllable and decoupled Gaussian digital twins directly from monocular ego-view robotic videos. Our method introduces a Graph-Structured Disentangled Representation that separates the robot, objects, and background into independently optimizable Gaussian subfields organized within a scene graph. To ensure stability, we propose a Task-Oriented Spatio-Temporal Alignment module that leverages the inherent logic of manipulation tasks-alternating between Motion and Skill phases-to construct accurate pseudo-ground-truth trajectories. Finally, a joint photometric-geometric optimization ensures the reconstructed scenes are temporally coherent, physically consistent, and simulation-ready. Extensive experiments demonstrate that our approach reconstructs interaction-driven dynamic scenes with high fidelity and controllability, effectively supporting downstream robotic tasks and policy learning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
