arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2610.29997cs.ROcs.CV
收藏

DONIC:基于人类演示的通用灵巧双手跟踪控制

DONIC: Generalist Dexterous Bimanual Tracking Control from Human Demonstrations

Mingze Sun, Ziang Cao, Yuan Fei, Qiankuan Tao, Zihan Wang, Tianqi Liu, Junhao Chen, Haiwen Diao, Fangzhou Hong, Zhaoxi Chen, Ziwei Liu

AI总结:

该研究针对现有灵巧跟踪模型未覆盖双手交互的问题,提出通用双手跟踪器DONIC及对应数据集BiHOI,其单策略在未微调时的跟踪成功率远超现有基线。

摘要:

尽管灵巧参考跟踪领域近期取得了进展,但现有模型主要聚焦于单手操作及有限范围的交互,未解决现实生活中常见的灵巧双手行为问题。接触丰富的双手操作数据稀缺,且接触协调的复杂性提升,使得开发通用双手跟踪器极具挑战性。因此,我们提出:1)DONIC,首个跨操作类别和人类运动源联合训练的通用双手跟踪器;2)BiHOI,包含1575个序列、覆盖229个物体的多视角数据集,重点关注持续的双手交互,其中重建的交互被重定向为机器人参考。DONIC将近端策略优化与行为克隆结合在反馈循环中,该循环将已验证的成功执行回收为训练演示。以物体为中心的逐手奖励维持物体运动反馈,同时根据每只手的参考接触角色对其进行监督。以剪辑为中心的优势函数消除了依赖参考的优势偏移,同时保留轨迹内差异。这些设计共同使单个策略能够学习跨异构参考的协调操作。无需微调,单个检查点在训练物体的未见过轨迹上达到86.79%的成功率,在保留物体上达到57.32%的成功率,分别超越最先进的双手跟踪基线77.36和45.73个百分点。

英文摘要:

Despite recent progress in dexterous reference tracking, existing models focus primarily on single-hand manipulation and a limited range of interactions, leaving the bimanual dexterous behaviors common in real life unaddressed. The scarcity of contact-rich bimanual manipulation data and the increased complexity of coordinating contacts make developing a generalist bimanual tracker particularly challenging. Therefore, we present 1) DONIC, the first generalist bimanual tracker jointly trained across operation categories and human-motion sources, together with 2) BiHOI, a multi-view dataset of 1,575 sequences spanning 229 objects and emphasizing sustained bimanual interaction, where reconstructed interactions are retargeted into robot references. DONIC combines proximal policy optimization with behavioral cloning in a feedback loop that recycles verified successful executions into training demonstrations. An object-centered per-hand reward maintains object-motion feedback while supervising each hand according to its reference contact role. Clip-Centered Advantages remove reference-dependent advantage offsets while preserving within-trajectory differences. Together, these designs enable one single policy to learn coordinated manipulation across heterogeneous references. Without finetuning, a single checkpoint achieves 86.79% success on unseen trajectories of training objects and 57.32% on held-out objects, surpassing the state-of-the-art bimanual tracking baselines by 77.36 and 45.73 percentage points.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑