DONIC:基于人类演示的通用灵巧双手跟踪控制
DONIC: Generalist Dexterous Bimanual Tracking Control from Human Demonstrations
AI总结:
该研究针对现有灵巧跟踪模型未覆盖双手交互的问题,提出通用双手跟踪器DONIC及对应数据集BiHOI,其单策略在未微调时的跟踪成功率远超现有基线。
摘要:
尽管灵巧参考跟踪领域近期取得了进展,但现有模型主要聚焦于单手操作及有限范围的交互,未解决现实生活中常见的灵巧双手行为问题。接触丰富的双手操作数据稀缺,且接触协调的复杂性提升,使得开发通用双手跟踪器极具挑战性。因此,我们提出:1)DONIC,首个跨操作类别和人类运动源联合训练的通用双手跟踪器;2)BiHOI,包含1575个序列、覆盖229个物体的多视角数据集,重点关注持续的双手交互,其中重建的交互被重定向为机器人参考。DONIC将近端策略优化与行为克隆结合在反馈循环中,该循环将已验证的成功执行回收为训练演示。以物体为中心的逐手奖励维持物体运动反馈,同时根据每只手的参考接触角色对其进行监督。以剪辑为中心的优势函数消除了依赖参考的优势偏移,同时保留轨迹内差异。这些设计共同使单个策略能够学习跨异构参考的协调操作。无需微调,单个检查点在训练物体的未见过轨迹上达到86.79%的成功率,在保留物体上达到57.32%的成功率,分别超越最先进的双手跟踪基线77.36和45.73个百分点。
英文摘要:
Despite recent progress in dexterous reference tracking, existing models focus primarily on single-hand manipulation and a limited range of interactions, leaving the bimanual dexterous behaviors common in real life unaddressed. The scarcity of contact-rich bimanual manipulation data and the increased complexity of coordinating contacts make developing a generalist bimanual tracker particularly challenging. Therefore, we present 1) DONIC, the first generalist bimanual tracker jointly trained across operation categories and human-motion sources, together with 2) BiHOI, a multi-view dataset of 1,575 sequences spanning 229 objects and emphasizing sustained bimanual interaction, where reconstructed interactions are retargeted into robot references. DONIC combines proximal policy optimization with behavioral cloning in a feedback loop that recycles verified successful executions into training demonstrations. An object-centered per-hand reward maintains object-motion feedback while supervising each hand according to its reference contact role. Clip-Centered Advantages remove reference-dependent advantage offsets while preserving within-trajectory differences. Together, these designs enable one single policy to learn coordinated manipulation across heterogeneous references. Without finetuning, a single checkpoint achieves 86.79% success on unseen trajectories of training objects and 57.32% on held-out objects, surpassing the state-of-the-art bimanual tracking baselines by 77.36 and 45.73 percentage points.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。