生成、迁移、适应:从单个人示范学习功能性灵巧抓取
Generate, Transfer, Adapt: Learning Functional Dexterous Grasping from a Single Human Demonstration
AI总结:
CorDex通过单个人示范生成合成数据,结合多模态预测网络和局部-全局融合模块,实现对新型物体的灵巧抓取学习。
摘要:
功能性灵巧机械手抓取是实现工具使用和复杂操作的关键能力,然而进展受到两个持续瓶颈的限制:大规模数据集的匮乏和学习模型中缺乏集成的语义和几何推理。在本工作中,我们提出了CorDex框架,该框架能够从仅一个人示范生成的合成数据中稳健地学习新型物体的灵巧功能性抓取。我们的方法核心是一种基于对应关系的数据引擎,该引擎在仿真中生成多样、高质量的训练数据。基于人类示范,我们的数据引擎生成同一类别的多种物体实例,通过对应估计将专家抓取迁移到生成的物体上,并通过优化进行适应。基于生成的数据,我们引入了多模态预测网络,整合了视觉和几何信息。通过设计局部-全局融合模块和重要性感知采样机制,我们实现了对功能性灵巧抓取的稳健且计算高效的预测。通过在各种物体类别上的广泛实验,我们证明CorDex能够很好地泛化到未见过的物体实例,并显著优于最先进的基线方法。
英文摘要:
Functional grasping with dexterous robotic hands is a key capability for enabling tool use and complex manipulation, yet progress has been constrained by two persistent bottlenecks: the scarcity of large-scale datasets and the absence of integrated semantic and geometric reasoning in learned models. In this work, we present CorDex, a framework that robustly learns dexterous functional grasps of novel objects from synthetic data generated from just a single human demonstration. At the core of our approach is a correspondence-based data engine that generates diverse, high-quality training data in simulation. Based on the human demonstration, our data engine generates diverse object instances of the same category, transfers the expert grasp to the generated objects through correspondence estimation, and adapts the grasp through optimization. Building on the generated data, we introduce a multimodal prediction network that integrates visual and geometric information. By devising a local-global fusion module and an importance-aware sampling mechanism, we enable robust and computationally efficient prediction of functional dexterous grasps. Through extensive experiments across various object categories, we demonstrate that CorDex generalizes well to unseen object instances and significantly outperforms state-of-the-art baselines.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。