Touch2Robot：人类演示循环中的机器人触觉
Touch2Robot: Robot Touch in the Human Demonstration Loop
浏览论文内容
中文总结 AI 辅助
提出Touch2Robot框架，在人类演示时可视化目标机器人手的接触，引导RL策略复现物体运动并偏好一致触觉，将行为蒸馏为实时重定向器。在四个真实任务中，将重放完成率从37.9%提升至72.1%，收集时间减少至18.2秒，并提升下游扩散策略性能29.1个百分点。
中文摘要 AI 辅助
人类演示提供了一种可扩展的收集操作数据的方式，但其接触在转移到机器人手时可能不稳定或不可行。直接在目标机器人上收集演示避免了这种不匹配，但大幅增加了数据收集成本。为解决这一权衡，我们提出了Touch2Robot框架，该框架让人类在收集演示的同时看到目标机器人手将如何接触物体。我们捕获人类手部运动、触觉手套测量和物体运动在人类操作过程中的数据。这些记录引导特定于物体的强化学习策略重现演示的物体运动，同时偏好与记录的人类触觉一致的接触。我们将学习到的行为提炼成一个统一的实时重定向器，将传入的人类观察和物体几何映射到机器人手配置。在收集过程中，预测的机器人配置与模拟中跟踪的物体姿态同步，以重建机器人-物体接触，这些接触被可视化以帮助演示者调整后续交互以适应目标手。在四个真实世界任务中，Touch2Robot将平均真实机器人重放完成率从37.9%提高到72.1%，相对于仅视觉反馈，同时将每次重放成功演示的收集时间从58.6秒减少到18.2秒。重建的目标手接触对真实机器人触觉测量达到44.2%的F1分数，基于Touch2Robot演示训练的策略在下游扩散策略性能上比仅视觉反馈提高29.1个百分点。这些结果表明，将机器人触觉引入人类演示循环提高了可扩展灵巧数据收集的质量和效率。项目网页：此URL。
英文摘要
Human demonstrations offer a scalable way to collect manipulation data, but their contacts may be unstable or infeasible when transferred to a robot hand. Collecting demonstrations directly on the target robot avoids this mismatch but substantially increases the cost of data collection. To address this trade-off, we present Touch2Robot, a framework that lets humans collect demonstrations while seeing how the target robot hand would contact the object. We capture human hand motion, tactile-glove measurements, and object motion during human manipulation. These recordings guide object-specific RL policies to reproduce the demonstrated object motion while favoring contacts consistent with the recorded human touch. We distill the learned behaviors into a unified real-time retargeter that maps incoming human observations and object geometry to robot hand configurations. During collection, the predicted robot configuration is synchronized with the tracked object pose in simulation to reconstruct robot-object contacts, which are visualized to help the demonstrator adapt subsequent interactions to the target hand. Across four real-world tasks, Touch2Robot improves average real-robot replay completion from 37.9% to 72.1% over visual-only feedback, while reducing the collection time per replay-successful demonstration from 58.6s to 18.2s. Reconstructed target-hand contacts achieve 44.2% F1 against real-robot tactile measurements, and policies trained on Touch2Robot demonstrations improve downstream Diffusion Policy performance by 29.1 percentage points over visual-only feedback. These results show that bringing robot touch into the human demonstration loop improves both the quality and efficiency of scalable dexterous data collection. Project webpage: https://Touch2Robot.github.io/.
发表机构
- Beijing Institute for General Artificial Intelligence (BIGAI)（北京通用人工智能研究院）
- ShanghaiTech University（上海科技大学）
- Shanghai Jiao Tong University（上海交通大学）
- Beihang University（北京航空航天大学）
机构由 AI 辅助整理，请以论文原文为准。
