NavGPT-2：释放大型视觉-语言模型的导航推理能力
NavGPT-2: Unleashing Navigational Reasoning Capability for Large Vision-Language Models
AI总结：
本研究提出NavGPT-2，通过冻结LLM对齐视觉内容并融合导航策略网络，弥合VLN专业模型与LLM导航范式差距，实现高效动作预测与导航推理，提升数据效率并达到专家水平。
摘要：
利用大型语言模型（LLMs）的显著进步，目前正兴起利用LLMs进行指令跟随机器人导航的倡议。这一趋势凸显了LLMs在泛化导航推理和多样化语言理解方面的潜力。然而，在视觉-语言导航（VLN）任务中集成LLMs时，观察到代理性能与先前下游专家模型相比存在显著差异。此外，语言在代理交互中解释和促进沟通的固有容量在这些集成中往往未被充分利用。在这项工作中，我们致力于弥合VLN专业模型与基于LLM的导航范式之间的鸿沟，同时保持LLMs在生成语言导航推理方面的解释能力。通过在冻结的LLM中对齐视觉内容，我们为LLMs涵盖视觉观察理解，并探索一种将LLMs与导航策略网络相结合的方法，以实现有效的动作预测和导航推理。我们展示了所提出方法的数据效率，并消除了基于LM的代理与最先进的VLN专家之间的差距。
英文摘要：
Capitalizing on the remarkable advancements in Large Language Models (LLMs), there is a burgeoning initiative to harness LLMs for instruction following robotic navigation. Such a trend underscores the potential of LLMs to generalize navigational reasoning and diverse language understanding. However, a significant discrepancy in agent performance is observed when integrating LLMs in the Vision-and-Language navigation (VLN) tasks compared to previous downstream specialist models. Furthermore, the inherent capacity of language to interpret and facilitate communication in agent interactions is often underutilized in these integrations. In this work, we strive to bridge the divide between VLN-specialized models and LLM-based navigation paradigms, while maintaining the interpretative prowess of LLMs in generating linguistic navigational reasoning. By aligning visual content in a frozen LLM, we encompass visual observation comprehension for LLMs and exploit a way to incorporate LLMs and navigation policy networks for effective action predictions and navigational reasoning. We demonstrate the data efficiency of the proposed methods and eliminate the gap between LM-based agents and state-of-the-art VLN specialists.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
