Splat-MOVER：基于可编辑高斯泼溅的多阶段开放词汇机器人操作
Splat-MOVER: Multi-Stage, Open-Vocabulary Robotic Manipulation via Editable Gaussian Splatting
AI总结：
提出Splat-MOVER模块化机器人技术栈，利用高斯泼溅的可编辑性，通过ASK-Splat提取语义与抓取可供性、SEE-Splat实时编辑场景数字孪生、Grasp-Splat生成候选抓取，实现了开放词汇的多阶段机器人操作任务。
摘要：
我们提出了Splat-MOVER，一个用于开放词汇机器人操作的模块化机器人技术栈，它利用高斯泼溅（GSplat）场景表示的可编辑性来实现多阶段操作任务。Splat-MOVER包含三个模块：(i) ASK-Splat，一种将语义和抓取可供性特征提取到3D场景中的GSplat表示。ASK-Splat实现了对3D场景的几何、语义和可供性理解，这在许多机器人任务中至关重要；(ii) SEE-Splat，一个实时场景编辑模块，使用3D语义掩码和填充来可视化机器人交互导致的物体运动。SEE-Splat在整个操作任务过程中创建了一个不断演化环境的“数字孪生”；(iii) Grasp-Splat，一个抓取生成模块，利用ASK-Splat和SEE-Splat为开放世界物体提出可供性对齐的候选抓取。ASK-Splat在操作前通过简短的扫描阶段从RGB图像中实时训练，而SEE-Splat和Grasp-Splat在操作期间实时运行。在Kinova机器人的硬件实验中，我们证明了Splat-MOVER在4个单阶段开放词汇操作任务和4个多阶段操作任务中，相较于两个最新基线方法表现出更优越的性能，其利用编辑后的场景来反映先前操作阶段引起的变化，这是现有基线方法无法实现的。项目演示视频和代码可在 https://splatmover.github.io 获取。
英文摘要：
We present Splat-MOVER, a modular robotics stack for open-vocabulary robotic manipulation, which leverages the editability of Gaussian Splatting (GSplat) scene representations to enable multi-stage manipulation tasks. Splat-MOVER consists of: (i) ASK-Splat, a GSplat representation that distills semantic and grasp affordance features into the 3D scene. ASK-Splat enables geometric, semantic, and affordance understanding of 3D scenes, which is critical in many robotics tasks; (ii) SEE-Splat, a real-time scene-editing module using 3D semantic masking and infilling to visualize the motions of objects that result from robot interactions in the real-world. SEE-Splat creates a "digital twin" of the evolving environment throughout the manipulation task; and (iii) Grasp-Splat, a grasp generation module that uses ASK-Splat and SEE-Splat to propose affordance-aligned candidate grasps for open-world objects. ASK-Splat is trained in real-time from RGB images in a brief scanning phase prior to operation, while SEE-Splat and Grasp-Splat run in real-time during operation. We demonstrate the superior performance of Splat-MOVER in hardware experiments on a Kinova robot compared to two recent baselines in four single-stage, open-vocabulary manipulation tasks and in four multi-stage manipulation tasks, using the edited scene to reflect changes due to prior manipulation stages, which is not possible with existing baselines. Video demonstrations and the code for the project are available at https://splatmover.github.io.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
