TacOT：通过触觉引导的最优传输从人类演示中学习接触丰富的灵巧操作
TacOT: Learning Contact-Rich Dexterous Manipulation from Human Demonstrations via Tactile-Guided Optimal Transport
浏览论文内容
中文总结 AI 辅助
本文提出TacOT框架，利用触觉引导的最优传输对齐人类演示与机器人操作，解决接触丰富灵巧操作中的人-机器人对应问题，在真实任务中显著提升成功率。
中文摘要 AI 辅助
从人类演示中学习接触丰富的灵巧操作为交互数据提供了可扩展的来源，然而由于不可靠的人-机器人对应关系，将这些技能转移到机器人上仍然具有挑战性。现有的人到机器人迁移方法通常依赖于视觉外观或运动相似性，这可能会将相似的运动与不同的接触状态和力模式关联起来。触觉动力学提供了交互感知的线索，以区分具有相似运动但不同接触状态的操作过程。我们引入了触觉引导的最优传输（TacOT），这是一个用于人到机器人接触丰富操作的框架。TacOT利用动作-触觉动态时间规整来识别具有一致交互动力学的人-机器人演示对应关系，并使用这些对应关系在共享策略表示空间中引导软最优传输对齐。这使得人类演示能够为机器人策略学习提供接触丰富的监督，而无需预定义的帧级人-机器人配对。在四个真实世界的灵巧操作任务中，TacOT在分布内任务上将闭环成功率比动作引导的OT提高了最多17个百分点，在针对性的到机器人分布外迁移下提高了20个百分点。进一步的分析表明，触觉引导的对应关系选择了具有更一致接触动力学的演示对，并产生了更好地反映交互状态演化的潜在表示。这些结果表明，触觉动力学为在接触丰富的灵巧操作中建立可靠的人-机器人对应关系提供了有效的语义信号。
英文摘要
Learning contact-rich dexterous manipulation from human demonstrations provides a scalable source of interaction data, yet transferring such skills to robots remains challenging due to unreliable human--robot correspondence. Existing human-to-robot transfer methods typically rely on visual appearance or motion similarity, which may associate similar motions with different contact states and force patterns. Tactile dynamics provide interaction-aware cues to distinguish manipulation processes with similar motions but different contact states. We introduce tactile-guided optimal transport (TacOT), a framework for human-to-robot contact-rich manipulation. TacOT leverages action--tactile dynamic time warping to identify human--robot demonstration correspondences with consistent interaction dynamics and uses these correspondences to guide soft optimal transport alignment in a shared policy representation space. This enables human demonstrations to provide contact-rich supervision for robot policy learning without requiring predefined frame-level human--robot pairing. Across four real-world dexterous manipulation tasks, TacOT improves closed-loop success rates over action-guided OT by up to 17 points on in-distribution tasks and 20 points under targeted human-to-robot out-of-distribution transfer. Further analyses show that tactile-guided correspondence selects demonstration pairs with more consistent contact dynamics and produces latent representations that better reflect interaction-state evolution. These results demonstrate that tactile dynamics provide an effective semantic signal for establishing reliable human-to-robot correspondence in contact-rich dexterous manipulation.
发表机构
- Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）
- School of Artificial Intelligence, Shanghai Jiao Tong University（上海交通大学人工智能学院）
- Xspark AI
- School of Mechanical and Aerospace Engineering, Nanyang Technological University（南洋理工大学机械与宇航工程学院）
机构由 AI 辅助整理，请以论文原文为准。
