通过像素引导导航技能桥接零样本物体导航与基础模型
Bridging Zero-shot Object Navigation and Foundation Models through Pixel-Guided Navigation Skill
AI总结：
本文提出像素引导导航技能PixNav，以像素为目标规范将基础模型与具身导航任务桥接，并结合LLM规划器实现长时程物体导航，在模拟和真实环境中验证了有效性。
摘要：
零样本物体导航对于家庭辅助机器人而言是一项具有挑战性的任务。该任务强调视觉定位、常识推理和移动能力，其中前两项是基础模型固有的能力。但在移动能力方面，大多数工作仍依赖于基于地图的规划方法。RGB空间与地图空间之间的差距使得将基础模型的知识直接迁移到导航任务变得困难。在这项工作中，我们提出了一种像素引导导航技能（PixNav），它弥合了基础模型与具身导航任务之间的差距。对于近期的基础模型来说，通过像素来指示物体是直接的，而以像素作为目标规范，我们的方法成为一种面向各种不同类型物体的通用导航策略。此外，我们的PixNav是一种纯基于RGB的策略，可以降低家庭辅助机器人的成本。实验证明了PixNav的鲁棒性，在局部路径规划任务中达到了80%以上的成功率。为了执行长时程物体导航，我们设计了一个基于LLM的规划器，利用物体与房间之间的常识知识来选择最佳航点。在逼真的室内模拟器和真实世界环境中的评估验证了我们所提出的导航策略的有效性。代码和视频演示可在 https://github.com/wzcai99/Pixel-Navigator 获取。
英文摘要：
Zero-shot object navigation is a challenging task for home-assistance robots. This task emphasizes visual grounding, commonsense inference and locomotion abilities, where the first two are inherent in foundation models. But for the locomotion part, most works still depend on map-based planning approaches. The gap between RGB space and map space makes it difficult to directly transfer the knowledge from foundation models to navigation tasks. In this work, we propose a Pixel-guided Navigation skill (PixNav), which bridges the gap between the foundation models and the embodied navigation task. It is straightforward for recent foundation models to indicate an object by pixels, and with pixels as the goal specification, our method becomes a versatile navigation policy towards all different kinds of objects. Besides, our PixNav is a pure RGB-based policy that can reduce the cost of home-assistance robots. Experiments demonstrate the robustness of the PixNav which achieves 80+% success rate in the local path-planning task. To perform long-horizon object navigation, we design an LLM-based planner to utilize the commonsense knowledge between objects and rooms to select the best waypoint. Evaluations across both photorealistic indoor simulators and real-world environments validate the effectiveness of our proposed navigation strategy. Code and video demos are available at https://github.com/wzcai99/Pixel-Navigator.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
