arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2610.41336cs.RO
收藏

HandMimicX:通过以物体为中心的策略从第一视角视频学习类人灵巧操作

HandMimicX: Learning Human-Like Dexterous Manipulation via an Object-Centric Policy from Egocentric Videos

Tsinghua University(清华大学);TsingTec(清芯科技);MeiTuan(美团);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));NVIDIA(英伟达)

Zihan Gao, Yufei Liu, Yuhong Zhang, Zhen Yu, Xinhao Huang, Zhinan You, Kai W. Ho, Zipeng Ye, Peng Jiao, Haoqian Wang

AI总结:

本研究提出HandMimicX框架,通过以物体为中心的策略从第一视角RGB-D视频学习类人灵巧操作,经仿真与真实机器人实验验证,可实现高效学习及技能的空间泛化迁移。

摘要:

从人类视频中学习灵巧操作仍是一个开放性挑战,尤其是在将单次演示迁移为空间可泛化的机器人技能,以及维持物理上合理的类人交互方面。与以往强调手部动作模仿的工作不同,我们提出一种以物体为中心的框架,用于从人类第一视角RGB-D视频中获取灵巧技能。在该框架内,我们根据物体运动制定强化学习(RL)目标,以将基础技能与特定手部轨迹解耦,并对手-物交互状态施加约束,以促进更符合解剖学的手部构型。在仿真环境与真实机器人上开展的大量实验表明,我们的方法可实现从人类视频的高效学习,以及在新空间构型上的可泛化技能迁移。

英文摘要:

Learning dexterous manipulation from human videos remains an open challenge, particularly in the context of transferring a single demonstration into a spatially generalizable robotic skill and maintaining physically plausible human-like interactions. Unlike prior works that emphasize hand-motion imitation, we introduce an object-centric framework for acquiring dexterous skills from egocentric human RGB-D videos. Within our framework, we formulate the reinforcement learning (RL) objective in terms of object motion to decouple the underlying skill from the specific hand trajectory and impose constraints on hand-object interaction states to promote more anatomically natural hand configurations. Extensive experiments in simulation and on real robots demonstrate that our method enables efficient learning from human videos and generalizable skill transfer across novel spatial configurations.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑