发表机构
Stanford University; MIT; Scale AI（斯坦福大学; 麻省理工学院; Scale AI公司）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究提出SPD仿真预训练框架，采集75小时多任务灵巧操作数据预训练因果Transformer，经微调后在真实任务中优于从头训练的行为克隆策略，为真实世界灵巧操作提供可行预训练来源。
AI 中文摘要
大规模预训练已显著提升机器人策略微调的数据效率，但这类进展主要依赖围绕简单平行夹爪构建的数据集和机器人本体。多手指灵巧手则面临数据匮乏的问题，因为真实遥操作的规模化成本高昂，而人类手部视频属于非对应本体数据，需要进行有损姿态估计和重定向。本文提出**仿真灵巧性预训练框架（Simulation Pre-training for Dexterity, SPD）**，这是一种完全利用仿真环境中采集的数据进行灵巧操作预训练的框架。在SPD中，人类通过VR头显操纵虚拟物体，可采集到对应本体的轨迹且无需机器人参与。借助5名操作人员，我们在一周内采集了75小时的多任务灵巧操作数据，并以此为基础在序列建模目标上预训练因果Transformer模型。我们通过在56自由度双手机器人灵巧装置上对1-2小时的真实演示数据进行微调，研究仿真预训练在真实任务中的优势。结果表明，我们的方法优于从头训练行为克隆策略，证明仿真遥操作是真实世界灵巧操作的可行预训练来源。我们还开展了消融研究，评估历史条件和短动作块对反应式控制的作用。
英文摘要
Large-scale pre-training has made robot policy fine-tuning increasingly data-efficient, but this progress has largely been driven by datasets and embodiments built around simple parallel-jaw grippers. Dexterous, multi-fingered hands remain comparatively data-starved because real teleoperation is costly to scale, while human hand video is off-embodiment and requires lossy pose estimation and retargeting. We introduce Simulation Pre-training for Dexterity (SPD), a pre-training framework for dexterous manipulation that uses data entirely collected in simulation. In SPD, humans manipulate virtual objects inside a VR headset, enabling on-embodiment trajectories and robot-free collection. With the help of five operators, we collect 75 hours of multi-task dexterous manipulation over one week, and use it to pre-train a causal transformer on a sequence modeling objective. We study the benefits of simulation pre-training on real-world tasks by fine-tuning on 1-2 hours of physical demonstrations on a 56-DoF bimanual dexterous setup. We find that our approach outperforms training behavior cloning policies from scratch, showing that simulation teleoperation is a viable pre-training source for real-world dexterous manipulation. We perform ablation studies, measuring the benefits of history conditioning and short action chunks for reactive control.
CommentsProject page: https://spd.bot
