AffordTrajDP：用于机器人操纵的动态 affordance 引导视觉运动策略学习

中文摘要 AI 辅助

affordance 引导的模仿学习通过将视觉感知压缩为任务特定的几何约束（例如固定接触点），在机器人操纵任务中展现出了令人印象深刻的性能。然而，常用的静态 affordance 在精度要求高的任务中或物体位置扰动下会出现精度不一致的问题，进而导致接触后轨迹漂移。为解决这一问题，我们提出了 AffordTrajDP，这是一个动态框架，它通过以物体为中心的时间传播构建 affordance 轨迹，以指导渐进式操纵过程。具体而言，给定 RGB-D 观测，我们的核心见解是，检索到的锚点 affordance（其捕获了末端执行器与目标物体之间的期望接触点）可以通过 affordance 传播，利用物体的 SE(3) 位姿作为天然传播介质向前传播，从而生成一条 affordance 轨迹，在整个执行过程中提供时间一致、感知状态的引导。AffordTrajDP 在 ManiSkill3 上实现了 70.0% 的平均成功率，比强大的基线方法高出最多 17.8%。在 Galaxea A1 和 UR7e 机械臂上进行的现实世界实验，涵盖了 StackCube、PickCup、AdapterInsertion、Ring-on-Peg、Put-in-Bowl 和 USB Insertion 任务，进一步验证了其在物体放置变化和外观变化下的鲁棒性，在 Galaxea A1 上评估了可见和不可见的物体实例， ablation 实验也证实了每个提出组件的贡献。

英文摘要

Affordance-guided imitation learning has shown impressive performance in robotic manipulation tasks by compressing visual perception into task-specific geometric constraints (e.g., fixed contact points). However, the commonly used static affordances can become inconsistent in precision-critical tasks or under object location perturbations, leading to post-contact trajectory drift. To address this issue, we propose AffordTrajDP, a dynamic framework that constructs affordance trajectories via object-centric temporal propagation to guide the progressive manipulation process. Specifically, given an RGB-D observation, our core insight is that a retrieved anchor affordance, which captures the desired contact point between the end-effector and the target object, can be propagated forward via affordance propagation, using the object's SE(3) pose as a natural propagation medium, to yield an affordance trajectory that provides temporally consistent, state-aware guidance throughout execution. AffordTrajDP achieves 70.0% average success rate on ManiSkill3, outperforming strong baselines by up to 17.8%. Real-world experiments on Galaxea A1 and UR7e robotic arms, covering StackCube, PickCup, AdapterInsertion, Ring-on-Peg, Put-in-Bowl, and USB Insertion, further validate robustness under object placement variations and appearance changes, with seen and unseen object instances evaluated on Galaxea A1, and ablations confirm the contribution of each proposed component.

发表机构

• Nanyang Technological University（南洋理工大学）

机构由 AI 辅助整理，请以论文原文为准。