Egocentric human videos offer a scalable alternative to costly robot demonstra- tions, yet transferring their manipulation priors to robot control remains chal- lenging. The commonly adopted retargeting-based approaches convert human wrist and finger poses into embodiment-specific action commands through hand- crafted geometric mappings. However, such mappings require meticulous recal- ibration across different embodiments and risk propagating hand-tracking errors into downstream robot control pipelines. To this end, we propose EgoJEPA, a three-stage framework that learns embodiment-agnostic representations from ego- centric human videos without rule-based human-to-robot retargeting: 1) Human- action pre-training learns effect-aware latent actions through JEPA-style visual prediction conditioned on hand-motion differences; 2) Latent-action-based world modeling jointly predicts future visual dynamics and latent actions, which decou- ples human prior pre-training from robot-specific control parameterizations; 3) Embodiment-specific tuning adapts the pretrained model to diverse robots through lightweight action adapters. Experiments on five real-world manipulation tasks across three robot embodiments demonstrate the effectiveness and efficiency of EgoJEPA. It achieves a 74% average success rate under the out-of-domain evalu- ation, compared with 65% for the retargeting baseline, while reducing cumulative cross-embodiment wall-clock cost by 47.2%, from 743.2 to 392.7 hours.
Analyzing Key Objectives in Human-to-Robot Retargeting for Dexterous Manipulation
Department of Automation, Beijing National Research Center for Information Science and Technology, Tsinghua University, China（自动化系，信息科学与技术国家工程研究中心，清华大学）
DexFlow: A Unified Approach for Dexterous Hand Pose Retargeting and Interaction
Institute of Computer Science, Wuhan University, Hubei, China（武汉大学计算机科学学院，湖北，中国）；Department of Mechanical Engineering, Massachusetts Institute of Technology（麻省理工学院机械工程系）；Department of Electrical and Computer Engineering, Georgia Institute of Technology（佐治亚理工学院电气与计算机工程系）；Institute of Technological Sciences of Wuhan University, Wuhan, Hubei Province, China（武汉大学技术科学研究院，武汉，湖北省，中国）
Human hand actions are quite complex, especially when they involve object manipulation, mainly due to the high dimensionality of the hand and the vast action space that entails. Imitating those actions with dexterous hand models involves different important and challenging steps: acquiring human hand information, retargeting it to a hand model, and learning a policy from acquired data. In this work, we capture the hand information by using a state-of-the-art hand pose estimator. We tackle the retargeting problem from the hand pose to a 29 DoF hand model by combining inverse kinematics and PSO …
DexTwist: Dexterous Hand Retargeting for Twist Motion via Mixed Reality-based Teleoperation
Autonomous Systems Lab, Institute of Computer Technology, TU Wien（自主系统实验室，计算机技术学院，维也纳技术大学）；Institute of Robotics and Mechatronics (DLR), German Aerospace Center（机器人与机电系统研究所（DLR），德国航空航天中心）
Tongji University（同济大学）；Shanghai Research Institute for Intelligent Autonomous Systems（上海自主智能无人系统研究院）；Purdue University（普渡大学）；Nanyang Technological University, Singapore（新加坡南洋理工大学）
ObjRetarget: An Object-Aware Motion Retargeting Framework with Anthropomorphic Arm Constraints and Polyhedral Hand Modeling
School of Artificial Intelligence, The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）人工智能学院）；School of Computing, University of Portsmouth（朴茨茅斯大学计算学院）
FunGrasp: Functional Grasping for Diverse Dexterous Hands
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；ETH Zürich（苏黎世联邦理工学院）；The Hong Kong University of Science and Technology（香港科技大学）
We propose to perform imitation learning for dexterous manipulation with multi-finger robot hand from human demonstrations, and transfer the policy to the real robot hand. We introduce a novel single-camera teleoperation system to collect the 3D demonstrations efficiently with only an iPad and a computer. One key contribution of our system is that we construct a customized robot hand for each user in the physical simulator, which is a manipulator resembling the same kinematics structure and shape of the operator's hand. This provides an intuitive interface and avoid unstable human-robot hand r…
TypeTele: Releasing Dexterity in Teleoperation by Dexterous Manipulation Types
School of Computer Science and Engineering, Sun Yat-sen University, China（中山大学计算机科学与工程学院）；Stanford University, USA（斯坦福大学）；Imperial College London, UK（伦敦帝国理工学院）
DexTele: A Dual-Arm Dexterous Teleoperation System Based on Motion Retargeting and Adaptive Force Control
School of Artificial Intelligence, The Chinese University of Hong Kong, Shenzhen（人工智能学院，香港中文大学（深圳））；School of Computing, University of Portsmouth（计算学院，朴茨茅斯大学）
EaDex: A Cross-Embodiment Dexterous Manipulation Framework from Low-Cost Demonstrations
Faculty of Robot Science and Engineering, Northeastern University（机器人科学与工程学院，东北大学）；Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（深圳先进技术研究所，中国科学院）；School of Automation, Nanjing University of Information Science and Technology（自动化学院，南京信息科学技术大学）
KineDex: Learning Tactile-Informed Visuomotor Policies via Kinesthetic Teaching for Dexterous Manipulation
Tongji University（同济大学）；Tsinghua University（清华大学）；Shanghai Jiao Tong University（上海交通大学）；University of Hong Kong（香港大学）；Shanghai Qi Zhi Institute（上海启智研究院）；Shanghai AI Lab（上海人工智能实验室）
FlowHOI: Flow-based Semantics-Grounded Generation of Hand-Object Interactions for Dexterous Robot Manipulation
Mohamed bin Zayed University of Artificial Intelligence（莫扎德·扎耶德人工智能大学）；Technical University of Munich（慕尼黑技术大学）；National University of Singapore（新加坡国立大学）；Westlake University（西湖大学）
