LeLaN：从真实场景视频中学习语言条件导航策略
LeLaN: Learning A Language-Conditioned Navigation Policy from In-the-Wild Videos
AI总结：
针对机器人需找到人类描述的任意物体的需求，提出LeLaN框架，利用大模型语义知识标注无标注无动作的真实场景视频数据训练导航策略，性能超现有最优方法且边缘推理速度达其4倍，并开源相关资源。
摘要：
世界上存在种类繁多的物体。要让机器人具备实用价值，它们需要拥有找到人类描述的任意物体的能力。本文提出LeLaN（学习语言条件导航策略），这是一种利用无标注、无动作的第一人称数据来学习可扩展的语言条件物体导航的新方法。LeLaN框架借助大型视觉-语言模型以及机器人基础模型的语义知识，对来自多种室内外环境的真实场景数据进行标注。我们标注了超过130小时在真实室内外环境中采集的数据，包括机器人观测数据、YouTube游览视频以及人类行走数据。超过1000次真实世界试验的大量实验表明，我们的方法能够利用无标注的无动作视频训练策略，该策略性能优于当前最优的机器人导航方法，同时在边缘计算设备上的推理速度是这些方法的4倍。我们在项目页面（https://learning-language-navigation.github.io/）开源了模型、数据集并提供补充视频。
英文摘要：
The world is filled with a wide variety of objects. For robots to be useful, they need the ability to find arbitrary objects described by people. In this paper, we present LeLaN(Learning Language-conditioned Navigation policy), a novel approach that consumes unlabeled, action-free egocentric data to learn scalable, language-conditioned object navigation. Our framework, LeLaN leverages the semantic knowledge of large vision-language models, as well as robotic foundation models, to label in-the-wild data from a variety of indoor and outdoor environments. We label over 130 hours of data collected in real-world indoor and outdoor environments, including robot observations, YouTube video tours, and human walking data. Extensive experiments with over 1000 real-world trials show that our approach enables training a policy from unlabeled action-free videos that outperforms state-of-the-art robot navigation methods, while being capable of inference at 4 times their speed on edge compute. We open-source our models, datasets and provide supplementary videos on our project page (https://learning-language-navigation.github.io/).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
