仿真与人类协同训练用于数据高效且可泛化的机器人操作
Sim-and-Human Co-training for Data-Efficient and Scene-Generalizable Bimanual Manipulation
AI总结：
SimHum通过结合仿真数据和人类数据，实现了数据高效且可泛化的机器人操作，实验表明其在现实任务中表现优于基线方法。
摘要：
合成仿真数据和现实世界的人类数据提供了可扩展的替代方案，以规避机器人数据收集的高昂成本。然而，这些来源分别面临仿真到现实的视觉差距和人类到机器人的身体差距，这限制了策略在现实世界场景中的泛化能力。在本工作中，我们识别出这两种来源之间自然但尚未被充分探索的互补性：仿真提供了人类数据所缺乏的机器人动作，而人类数据则提供了仿真难以渲染的现实世界观察。受这一洞察的启发，我们提出了SimHum，一种协同训练框架，同时从模拟的机器人动作中提取运动学先验和从现实世界的人类观察中提取视觉先验。基于这两种互补的先验，我们实现了在现实世界任务中的数据高效且可泛化的机器人操作。实验表明，SimHum在相同的数据收集预算下，比基线方法高出高达40%，并在仅使用80个真实数据的情况下，实现了62.5%的OOD成功率，比仅使用真实数据的基线方法高出7.1倍。视频和额外信息可在项目网站https://kaipengfang.github.io/sim-and-human找到。
英文摘要：
Real-robot demonstrations are prohibitively expensive, while simulation data and real-world human demonstrations are both scalable but each leaves a distinct gap: simulation suffers from a sim-to-real visual gap, and human data suffers from a human-to-robot embodiment gap. In this work, we identify a natural yet underexplored complementarity between these sources: simulation contributes robot-valid actions absent in human data, while human data provides real-world observations that simulation struggles to render. Building on this insight, we present SimHum, a co-training recipe that extracts kinematic priors from simulation and visual priors from human observations, then fine-tunes on a small real-robot dataset. SimHum exhibits strong scene-generalizable and data-efficient capabilities. With only 80 real-robot episodes per task, it achieves 62.5% success on held-out OOD scenes across four bimanual tabletop tasks, 53.7% higher than Real only in absolute success rate. Moreover, in a controlled data-collection study with matched collection time, SimHum improves over the best single-source pre-training baseline by 35.0% in absolute success rate. Project page: https://kaipengfang.github.io/sim-and-human/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
