指尖感知到本体感觉与主动动作的自监督锚定用于机器人模仿学习
Self-Supervised Anchoring of Fingertip Sensing to Proprioception and Proactive Actions for Robot Imitation Learning
浏览论文内容
中文总结 AI 辅助
针对模仿学习中指尖感知难以利用的问题，提出本体感觉与主动锚定（PROPRA）预训练方法，将指尖传感器历史与本体感觉和动作对齐，在真实操作任务上提升成功率。
中文摘要 AI 辅助
机器人模仿学习通常依赖外部摄像头，然而诸如物体接近、接触开始和抓取状态等局部交互线索，由于遮挡和时间分辨率有限，难以在指尖附近观察到。我们研究如何利用压敏触觉和反射式接近传感器，以及预训练的传感器编码器，有效地将互补的指尖感知融入模仿学习。这两种模态在不同的操作阶段提供信息：接近传感器在接触前提供信息，而触觉传感器在接触后变得有信息量。然而，简单地将这些信号添加到策略中并不能持续提高性能，甚至可能不如仅视觉的策略，这表明稀疏的、阶段相关的传感器信号难以从有限的演示中利用。因此，我们提出了一种本体感觉锚定的预训练方法，即本体感觉与主动锚定（PROPRA），该方法将每个指尖传感器历史与本体感觉和动作片段独立对齐。这提供了一个持续可用的感觉运动参考，使得每个传感器能够在其信息阶段独立对齐。在真实世界操作任务上的实验表明，我们的预训练方法相比仅视觉策略和图像锚定预训练基线，提高了平均成功率。表征分析进一步表明，它保留了关于接触前状态的更丰富信息，从而能够更有效地利用互补的指尖感知。请参考我们的项目页面：此 https URL
英文摘要
Robotic imitation learning often relies on external cameras, yet local interaction cues such as object proximity, contact onset, and grasp state are difficult to observe near the fingertips because of occlusion and limited temporal resolution. We study how to effectively incorporate complementary fingertip sensing into imitation learning using pressure-sensitive tactile and reflective proximity sensors, along with pretrained sensor encoders. The two modalities provide information at different manipulation phases: proximity sensing is informative before contact, whereas tactile sensing becomes informative after contact. However, naively adding these signals to a policy does not consistently improve performance and can even underperform vision-only policies, suggesting that sparse, phase-dependent sensor signals are difficult to exploit from limited demonstrations. We therefore propose a proprioception-anchored pretraining method, PROprioceptive-and-PRoactive Anchoring (PROPRA), which independently aligns each fingertip sensor history with proprioceptive and action segments. This provides a continuously available sensorimotor reference, allowing each sensor to be aligned independently during its informative phases. Experiments on real-world manipulation tasks show that our pretraining method improves average success rates over vision-only policies and image-anchored pretraining baselines. Representation analysis further shows that it preserves richer information about pre-contact states, enabling more effective use of complementary fingertip sensing. Please refer to our project page: https://tomohiromotoda.github.io/nia.propra/
发表机构
- National Institute of Advanced Industrial Science and Technology (AIST)（产业技术综合研究所（AIST））
- CNRS-AIST JRL (Joint Robotics Laboratory), IRL（法国国家科学研究中心-产业技术综合研究所联合机器人实验室（JRL），IRL）
机构由 AI 辅助整理，请以论文原文为准。
