Hand2Robot：面向机器人的、基于人类演示的以物体为中心的4D手部重建方法
Hand2Robot: Object-Centric 4D Hand Reconstruction from Human Demonstrations for Robotics
AI总结：
Hand2Robot是面向机器人的以物体为中心的4D手部重建框架，通过优化手-物交互与动作重定向生成机器人动作监督，可补充机器人数据以提升目标任务策略性能。
摘要：
人类演示无需使用目标机器人即可捕捉与任务相关的操作，但记录的手部动作无法直接作为机器人动作的监督信号。手部重建与动作重定向为获取此类监督信号提供了直接途径，然而仅恢复手部动作无法确保手-物交互的物理一致性。为解决该问题，我们提出Hand2Robot，一种用于机器人策略后训练的以物体为中心的4D手部重建框架。给定人类执行目标任务的视频，Hand2Robot会提取交互片段、重建物体几何结构，并通过手-物交互优化与动作重定向生成机器人动作监督信号。对于物体跟踪，我们将观测置信度与时间运动约束相结合，从逐帧位姿候选中选择全局一致的物体位姿轨迹；对于手部优化，我们以物体几何结构与运动为参考，结合分阶段2D对齐与物体局部接触约束，优化手部轨迹与手指关节配置。在DexYCB和HO3D-v3数据集上的实验显示，该方法提升了物体位姿精度与手-物交互质量；在三个真实机器人灵巧操作任务上的进一步实验表明，重建并重定向后的人类演示可有效补充有限的机器人数据，用于目标任务策略的后训练。
英文摘要：
Human demonstrations can capture task-relevant manipulation without using the target robot, but the recorded hand motions cannot directly serve as robot action supervision. Hand reconstruction and motion retargeting provide a direct route to such supervision, yet recovering hand motion alone does not ensure physically consistent hand-object interactions. To address this issue, we introduce Hand2Robot, an object-centric 4D hand reconstruction framework for robot policy post-training. Given videos of humans performing the target task, Hand2Robot extracts interaction clips, reconstructs object geometry, and generates robot action supervision through hand-object interaction optimization and motion retargeting. For object tracking, we combine observation confidence with temporal motion constraints to select a globally consistent object pose trajectory from per-frame pose candidates. For hand refinement, we use object geometry and motion as a reference and combine staged 2D alignment with object-local contact constraints to refine hand trajectories and finger joint configurations. Experiments on DexYCB and HO3D-v3 show improved object pose accuracy and hand-object interaction quality. Experiments on three real-robot dexterous manipulation tasks further show that reconstructed and retargeted human demonstrations can effectively supplement limited robot data for target-task policy post-training.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
