RealDex：迈向机器人灵巧手的类人抓取
RealDex: Towards Human-like Grasping for Robotic Dexterous Hand
- ShanghaiTech University（上海科技大学）
- The University of Hong Kong（香港大学）
- Texas A&M University（德克萨斯农工大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本研究提出融入人类行为模式的灵巧手抓取数据集RealDex，以及结合多模态大语言模型的类人抓取动作生成框架，实验验证其性能优越，将推动人形机器人真实场景操作能力发展。
AI中文摘要：
本文介绍了RealDex，这是一个开创性的数据集，捕获了融入人类行为模式的真实灵巧手抓取动作，并辅以多视角、多模态视觉数据。我们利用遥操作系统实现人机手部姿态的实时无缝同步。这类人动作数据集对于训练灵巧手更自然、精准地模仿人类动作至关重要，有望推动人形机器人在真实场景中实现自主感知、认知与操作。此外，我们提出了一种前沿的灵巧抓取动作生成框架，该框架契合人类经验，通过有效利用多模态大语言模型（Multimodal Large Language Models）提升了真实场景适用性。大量实验表明，我们的方法在RealDex及其他公开数据集上均表现出优越性能。完整数据集和代码将在本工作发表后公开。
英文摘要：
In this paper, we introduce RealDex, a pioneering dataset capturing authentic dexterous hand grasping motions infused with human behavioral patterns, enriched by multi-view and multimodal visual data. Utilizing a teleoperation system, we seamlessly synchronize human-robot hand poses in real time. This collection of human-like motions is crucial for training dexterous hands to mimic human movements more naturally and precisely. RealDex holds immense promise in advancing humanoid robot for automated perception, cognition, and manipulation in real-world scenarios. Moreover, we introduce a cutting-edge dexterous grasping motion generation framework, which aligns with human experience and enhances real-world applicability through effectively utilizing Multimodal Large Language Models. Extensive experiments have demonstrated the superior performance of our method on RealDex and other open datasets. The complete dataset and code will be made available upon the publication of this work.
