具身3D场景图用于开放世界移动操作
Articulated 3D Scene Graphs for Open-World Mobile Manipulation
AI总结：
本文提出MoMa-SG框架，通过语义-运动学3D场景图实现开放世界移动操作中物体的稳健交互与操控。
摘要：
语义学已经使3D场景理解和基于可行性的物体交互成为可能。然而，机器人在现实世界环境中面临一个关键限制：它们无法预测物体如何移动。长周期移动操作需要弥合语义、几何和运动学之间的差距。在本文中，我们提出了MoMa-SG，一种新的框架，用于构建包含大量可交互物体的具身场景图。给定包含多个物体运动的RGB-D序列，我们通过时间分割物体交互并利用遮挡鲁棒的点跟踪来推断物体运动。然后，我们将点轨迹提升到3D并使用一种新的统一扭估计公式来估计运动学模型，该公式在单次优化过程中稳健地估计旋转和直线关节参数。接下来，我们将物体与估计的运动学关联，并通过推理父-子关系在识别的打开状态中检测到的物体。我们还引入了新的Arti4D-Semantic数据集，该数据集唯一地结合了层次化的物体语义，包括父-子关系标签和物体轴注释，在62个真实世界RGB-D序列中包含600个物体交互和三种不同的观察范式。我们对两个数据集进行了广泛评估，并消除了我们方法的关键设计选择。此外，对四足机器人和移动机械臂的现实世界实验表明，我们的语义-运动学场景图能够使日常家庭环境中的具身物体操作更加稳健。我们提供代码和数据在：https://momasg.cs.uni-freiburg.de。
英文摘要：
Semantic scene understanding has enabled robots to reason and perform affordance-driven object interaction. Despite this, real-world manipulation remains fundamentally limited by the inability to predict how objects move and are articulated. Long-horizon mobile manipulation requires closing the gap between semantics, geometry, and kinematics. In this work, we present MoMa-SG, a novel framework for building semantic-kinematic 3D scene graphs of articulated scenes containing a myriad of interactable objects. Given RGB-D sequences containing multiple object articulations, we temporally segment object interactions and infer object motion using occlusion-robust point tracking. We lift point trajectories to 3D and estimate articulation models using a novel unified twist estimation formulation that robustly estimates revolute and prismatic joint parameters in a single optimization pass. Next, we associate objects with articulations and detect contained objects by reasoning over parent-child relations at identified opening states. We also introduce the Arti4D-Semantic dataset, which provides parent-child relation annotations between container and contained objects together with object axis annotations across 62 in-the-wild RGB-D sequences containing 600 object interactions and three distinct observation paradigms. We extensively evaluate the performance of MoMa-SG on two datasets and ablate key design choices of our approach. In addition, real-world experiments on both a quadruped and a mobile manipulator demonstrate robust manipulation of articulated objects in everyday home environments. We provide code and data at: https://momasg.cs.uni-freiburg.de.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
