灵巧点策略：从人类演示中学习基于点的灵巧手策略
Dexterous Point Policy: Learning Point-based Dexterous Hand Policies from Human Demonstrations
- KAIST（韩国科学技术院）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
提出Dexterous Point Policy框架，通过统一3D关键点表示从人类视频学习灵巧操作策略，无需机器人演示，在真实任务中达到75%成功率。
AI中文摘要：
基于人类演示视频预训练的机器人基础模型显示出潜力，但当策略部署到真实机器人时仍存在显著的具身差距。常见的补救措施是在机器人特定演示上微调这些模型。然而，机器人数据收集可能过于昂贵和耗时，这在灵巧操作中尤为突出，例如，即使是单个原子任务，遥操作多指手也可能需要数天。为了解决这个问题，我们引入了Dexterous Point Policy，一个直接从人类视频学习灵巧操作策略且无需机器人演示的框架。我们的核心见解是，统一的3D关键点表示在用于观察和动作时，可以桥接人类和机器人的具身。具体来说，我们从原始视频中提取任务相关物体和人类手的3D关键点，并训练一个自回归变换器来处理这些关键点。我们观察到，在关键点层面，特别是手腕和指尖，人类和机器人的行为紧密对齐，从而实现直接策略迁移。在一套包括拾取放置和工具使用的真实机器人任务中，Dexterous Point Policy达到了75.0%的成功率，而最先进的VLA基线仅达到1.0%。此外，我们的方法对未见过的场景具有很强的泛化能力，包括多物体环境和新型物体类别。
英文摘要：
Robotic foundation models pre-trained on human demonstration videos have shown promise, but a significant embodiment gap remains when the resulting policies are deployed on real robots. A common remedy is to fine-tune these models on robot-specific demonstrations. However, robot data collection can be prohibitively expensive and time-consuming, which is particularly acute in dexterous manipulation, e.g., teleoperating a multi-fingered hand for even a single atomic task can take days. To address this, we introduce Dexterous Point Policy, a framework that learns dexterous manipulation policies directly from human videos and requires no robot demonstrations. Our core insight is that a unified 3D keypoint representation can bridge human and robot embodiments when used for both observations and actions. Specifically, we extract 3D keypoints of task-relevant objects and human hands from raw videos, and train an autoregressive transformer over these keypoints. We observe that at the keypoint level, specifically the wrist and fingertips, human and robot behaviors closely align, enabling direct policy transfer. On a suite of real-robot tasks spanning pick-and-place and tool use, Dexterous Point Policy attains 75.0% success, whereas a state-of-the-art VLA baseline reaches only 1.0%. Furthermore, our method generalizes strongly to unseen scenarios, including multi-object environments and novel object categories.
