发表机构
Nanjing University; China Mobile Research Institute（南京大学; 中国移动研究院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
C2Dex是基于单目视频的灵巧操作框架，通过聚合物体侧稳定接触实现HOI重建与手物几何保留，在DexYCB、TACO上的轨迹成功率显著优于基线，且真实机器人复现可行。
AI 中文摘要
采集高质量的灵巧机器人操作演示成本高且难度大，而单目人类视频提供了多样化操作行为的可扩展来源。然而将此类演示迁移到灵巧机器人仍具挑战性：单目手物交互（HOI）重建常产生时间不稳定的接触和物理上不合理的交互，传统重定向方法难以在不同手部形态间保留任务相关接触和局部交互几何结构。本文提出C2Dex，这是一个围绕共享交互表示构建的视频转灵巧操作框架：通过在标准物体空间中聚合带噪声的逐帧观测，恢复稳定的物体侧接触。这些稳定接触兼具双重作用：一是作为轨迹级约束，引导重建得到时间一致且物理合理的人类HOI轨迹；二是作为灵巧手部的显式迁移目标，其中拉普拉斯交互优化保留不同形态间的局部手物几何，残差强化学习在仿真中优化轨迹。在DexYCB和TACO上的实验表明，C2Dex的端到端轨迹成功率分别为57.78%和26.67%，在相同评估标准下显著优于最强基线（17.78%和10.00%）。真实机器人复现实验进一步证明其在各类接触丰富的操作任务中的物理可行性。项目页面：this https URL
英文摘要
High-quality demonstrations for dexterous robot manipulation are costly and difficult to collect, whereas monocular human videos provide a scalable source of diverse manipulation behaviors. However, transferring such demonstrations to dexterous robots remains challenging: monocular hand-object interaction (HOI) reconstruction often produces temporally unstable contacts and physically implausible interactions, while conventional retargeting methods struggle to preserve task-relevant contacts and local interaction geometry across different hand embodiments. We present C2Dex, a video-to-dexterous-manipulation framework built around a shared interaction representation: stable object-side contacts recovered by aggregating noisy frame-wise observations in the canonical object space. These stable contacts serve a dual role: as trajectory-level constraints that guide reconstruction toward temporally coherent and physically plausible human HOI trajectories, and as explicit transfer targets for the dexterous hand, where Laplacian interaction optimization preserves the local hand-object geometry across embodiments and residual reinforcement learning refines the trajectory in simulation. Experiments on DexYCB and TACO show that C2Dex achieves end-to-end trajectory success rates of 57.78% and 26.67%, respectively, substantially outperforming the strongest baselines (17.78% and 10.00%) under identical evaluation criteria. Real-robot replay experiments further demonstrate physical feasibility across diverse contact-rich manipulation tasks. Project page: https://k-jie.github.io/C2Dex/
Comments9 pages, 5 figures. Submitted to IEEE Robotics and Automation Letters (RA-L). Project page: https://k-jie.github.io/C2Dex/
