基于自我中心视频与示范的机器人导航任务协同训练
Co-training with Ego-centric Video and Demonstration for Robot Navigation Task
AI总结：
提出将自我中心行走视频转化为移动机器人模仿学习数据集的框架，通过联合训练VLA模型提升语言理解和动作生成能力。
摘要：
视觉-语言-动作（VLA）模型在多种机器人任务中展现出潜力，但其性能严重依赖于大规模高质量训练数据，而在真实机器人上收集这些数据成本高昂且耗时。虽然先前的工作已经探索了利用自我中心人类视频来增强操作数据集，但由于运动过程中的视角变化，将此类方法应用于移动机器人导航仍然具有挑战性。在本文中，我们提出了一个框架，将自我中心行走视频转化为移动机器人模仿学习的数据集。该方法从人类视频中估计相机运动，并将其转换为与地面移动机器人兼容的动作表示。通过联合训练基于人类数据和机器人收集数据的VLA模型，该模型在语言理解和鲁棒动作生成方面比单独使用任一数据源训练取得了更好的性能。在水果搜索导航任务上的实验表明，人类自我中心视频为移动机器人学习提供了有效且可扩展的数据源。
英文摘要：
Vision-language-action (VLA) models are promising for diverse robotic tasks, but their performance heavily depends on large-scale high-quality training data, whose collection on real robots is costly and time-consuming. While prior work has explored augmenting manipulation datasets with egocentric human videos, applying such approaches to mobile robot navigation remains challenging due to viewpoint changes during locomotion. In this paper, we propose a framework that converts egocentric walking videos into datasets for mobile robot imitation learning. The proposed method estimates camera motion from human videos and transforms it into action representations compatible with ground mobile robots. By jointly training a VLA model on human-derived and robot-collected datasets, the model achieves improved language understanding and more robust action generation than training with either data source alone. Experiments on a fruit-search navigation task demonstrate that human egocentric videos provide an effective and scalable data source for mobile robot learning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
