MimicDreamer：对齐人类与机器人演示以实现可扩展的 VLA 训练
MimicDreamer: Aligning Human and Robot Demonstrations for Scalable VLA Training
AI总结：
本文提出 MimicDreamer 框架，通过视觉、视角与动作联合对齐，将低成本人类演示转换为机器人可用视频和动作监督，显著提升 VLA 模型少样本真实操作性能。
摘要：
Vision Language Action（VLA，视觉-语言-动作）模型的泛化能力来自多样化的训练数据，但收集具身机器人交互数据仍然极其昂贵。相比之下，人类演示视频的收集更具可扩展性且成本更低，近期研究也证实了其在训练 VLA 模型方面的有效性。然而，人类视频与机器人执行视频之间仍存在显著的域差距，包括不稳定的相机视角、人手与机械臂之间的视觉差异，以及运动动力学差异。为弥合这一差距，我们提出 MimicDreamer，这是一个通过联合对齐视觉、视角和动作，将快速、低成本的人类演示转化为机器人可用监督信号，从而直接支持策略训练的框架。在视觉对齐方面，我们提出 H2R Aligner，这是一种视频扩散模型，通过迁移人类操作镜头中的运动来生成高保真机器人演示视频。在视角稳定方面，我们提出 EgoStabilizer，它通过单应性变换将第一视角视频规范化，并对由扭曲造成的遮挡和失真进行修复。在动作对齐方面，我们将人手轨迹映射到机器人坐标系，并应用受约束的逆运动学求解器，以生成可行、低抖动且能精确跟踪位姿的关节指令。实验表明，仅使用我们合成的人到机器人视频训练的 VLA 模型即可在真实机器人上实现少样本执行。此外，与仅使用真实机器人数据训练的模型相比，利用人类数据扩展训练显著提升了性能；在六个具有代表性的操作任务上，我们的方法将平均成功率提高了 14.7%。
英文摘要：
Vision Language Action (VLA) models derive their generalization capability from diverse training data, yet collecting embodied robot interaction data remains prohibitively expensive. In contrast, human demonstration videos are far more scalable and cost-efficient to collect, and recent studies confirm their effectiveness in training VLA models. However, a significant domain gap persists between human videos and robot-executed videos, including unstable camera viewpoints, visual discrepancies between human hands and robotic arms, and differences in motion dynamics. To bridge this gap, we propose MimicDreamer, a framework that turns fast, low-cost human demonstrations into robot-usable supervision by jointly aligning vision, viewpoint, and actions to directly support policy training. For visual alignment, we propose H2R Aligner, a video diffusion model that generates high-fidelity robot demonstration videos by transferring motion from human manipulation footage. For viewpoint stabilization, EgoStabilizer is proposed, which canonicalizes egocentric videos via homography and inpaints occlusions and distortions caused by warping. For action alignment, we map human hand trajectories to the robot frame and apply a constrained inverse kinematics solver to produce feasible, low-jitter joint commands with accurate pose tracking. Empirically, VLA models trained purely on our synthesized human-to-robot videos achieve few-shot execution on real robots. Moreover, scaling training with human data significantly boosts performance compared to models trained solely on real robot data; our approach improves the average success rate by 14.7\% across six representative manipulation tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
