CityWalker：从网络规模视频中学习具身城市导航
CityWalker: Learning Embodied Urban Navigation from Web-Scale Videos
AI总结：
本文提出CityWalker，利用网络视频数据训练具身智能体进行城市导航，通过可扩展数据处理提取动作监督，实现大规模模仿学习，显著提升导航性能。
摘要：
在动态城市环境中导航对具身智能体提出了重大挑战，要求其具备高级空间推理能力并遵循常识规范。尽管已有进展，现有视觉导航方法在无地图或非街道场景中仍存在困难，限制了自主智能体（如最后一英里配送机器人）的部署。为克服这些障碍，我们提出了一种可扩展的、数据驱动的方法，通过使用从网络获取的数千小时野外城市步行和驾驶视频来训练智能体，实现类人城市导航。我们引入了一个简单且可扩展的数据处理流程，从这些视频中提取动作监督信号，从而无需昂贵的标注即可进行大规模模仿学习。我们的模型学习复杂的导航策略，以应对多样化的挑战和关键场景。实验结果表明，在大规模、多样化数据集上进行训练显著提升了导航性能，超越了现有方法。这项工作展示了利用丰富的在线视频数据为动态城市环境中的具身智能体开发稳健导航策略的潜力。项目主页：https://ai4ce.github.io/CityWalker/。
英文摘要：
Navigating dynamic urban environments presents significant challenges for embodied agents, requiring advanced spatial reasoning and adherence to common-sense norms. Despite progress, existing visual navigation methods struggle in map-free or off-street settings, limiting the deployment of autonomous agents like last-mile delivery robots. To overcome these obstacles, we propose a scalable, data-driven approach for human-like urban navigation by training agents on thousands of hours of in-the-wild city walking and driving videos sourced from the web. We introduce a simple and scalable data processing pipeline that extracts action supervision from these videos, enabling large-scale imitation learning without costly annotations. Our model learns sophisticated navigation policies to handle diverse challenges and critical scenarios. Experimental results show that training on large-scale, diverse datasets significantly enhances navigation performance, surpassing current methods. This work shows the potential of using abundant online video data to develop robust navigation policies for embodied agents in dynamic urban settings. Project homepage is at https://ai4ce.github.io/CityWalker/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
