WARPED：用于机器人策略学习的腕部对齐渲染
WARPED: Wrist-Aligned Rendering for Robot Policy Learning from Egocentric Human Demonstrations
AI总结：
WARPED通过单目RGB数据合成腕部视角观测，实现从第一人称人类示范数据中训练视觉运动策略，相比传统方法减少5-8倍的数据收集时间。
摘要：
最近在学习人类示范方面的进展在解决训练稳健视觉运动策略所需的数据可扩展性和高成本问题上显示出有前途的结果。然而，现有方法通常受限于多视角相机设置、深度传感器或定制硬件，通常仅限于从第三人称或第一人称相机执行策略。在本文中，我们提出了WARPED，一个框架，旨在从人类示范视频中合成逼真的腕部视角观测，以仅使用单目RGB数据训练视觉运动策略。通过从第一人称RGB相机收集的数据，我们的系统利用视觉基础模型来初始化交互场景。然后使用手-物体交互管道来跟踪手和被操作物体，并将轨迹重新定位到机械臂末端执行器。最后，通过高斯点划线合成逼真腕部视角观测，直接训练机器人策略。我们证明WARPED在五个桌面操作任务中实现的成功率与训练于遥控示范数据的策略相当，同时数据收集时间减少5-8倍。
英文摘要：
Recent advancements in learning from human demonstration have shown promising results in addressing the scalability and high cost of data collection required to train robust visuomotor policies. However, existing approaches are often constrained by a reliance on multiview camera setups, depth sensors, or custom hardware and are typically limited to policy execution from third-person or egocentric cameras. In this paper, we present WARPED, a framework designed to synthesize realistic wrist-view observations from human demonstration videos to facilitate the training of visuomotor policies using only monocular RGB data. With data collected from an egocentric RGB camera, our system leverages vision foundation models to initialize the interactive scene. A hand-object interaction pipeline is then employed to track the hand and manipulated object and retarget the trajectories to a robotic end-effector. Lastly, photo-realistic wrist-view observations are synthesized via Gaussian Splatting to directly train a robotic policy. We demonstrate that WARPED achieves success rates comparable to policies trained on teleoperated demonstration data for five tabletop manipulation tasks, while requiring 5-8x less data collection time.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
