SPOT:用于以对象为中心的操作的 SE(3) 位姿轨迹扩散
SPOT: SE(3) Pose Trajectory Diffusion for Object-Centric Manipulation
AI总结:
本文提出 SPOT 框架,以相对目标的 SE(3) 对象位姿轨迹表示任务并条件化扩散策略,支持多类型演示与跨本体泛化,仅用八个 iPhone 演示即完成全部真实任务。
摘要:
我们提出了 SPOT,一个以对象为中心的模仿学习框架。其核心思想是通过以对象为中心的表示来刻画每个任务,具体而言是相对于目标的 SE(3) 对象位姿轨迹。该方法将本体动作与感官输入解耦,便于从多种演示类型中学习,包括基于动作的人类手部演示和无动作的人类手部演示,以及跨本体泛化。此外,对象位姿轨迹能够固有地从演示中捕获规划约束,而无需人工制定规则。为引导机器人执行任务,对象轨迹被用于条件化一个扩散策略。我们在仿真和真实世界任务上系统评估了所提方法。在真实世界评估中,仅使用 iPhone 拍摄的八个演示,我们的方法便完成了所有任务,同时完全遵守任务约束。项目页面:https://nvlabs.github.io/object_centric_diffusion
英文摘要:
We introduce SPOT, an object-centric imitation learning framework. The key idea is to capture each task by an object-centric representation, specifically the SE(3) object pose trajectory relative to the target. This approach decouples embodiment actions from sensory inputs, facilitating learning from various demonstration types, including both action-based and action-less human hand demonstrations, as well as cross-embodiment generalization. Additionally, object pose trajectories inherently capture planning constraints from demonstrations without the need for manually-crafted rules. To guide the robot in executing the task, the object trajectory is used to condition a diffusion policy. We systematically evaluate our method on simulation and real-world tasks. In real-world evaluation, using only eight demonstrations shot on an iPhone, our approach completed all tasks while fully complying with task constraints. Project page: https://nvlabs.github.io/object_centric_diffusion
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。