See, Point, Fly：一种用于通用无人机导航的无学习VLM框架
See, Point, Fly: A Learning-Free VLM Framework for Universal Unmanned Aerial Navigation
AI总结：
SPF是一个免训练的VLM框架，将空中导航动作预测视为2D空间定位，通过迭代标注2D路点并转换为3D位移向量，实现通用无人机导航，在模拟和真实环境中大幅超越现有方法。
摘要：
我们提出了See, Point, Fly (SPF)，一种基于视觉语言模型（VLM）的免训练空中视觉语言导航（AVLN）框架。SPF能够根据任何类型的自由形式指令，在任何环境中导航到任何目标。与现有的将动作预测视为文本生成任务的基于VLM的方法不同，我们的关键见解是将AVLN的动作预测视为2D空间定位任务。SPF利用VLM将模糊的语言指令分解为对输入图像上2D路点的迭代标注。结合预测的行驶距离，SPF将预测的2D路点转换为3D位移向量，作为无人机的动作指令。此外，SPF还自适应地调整行驶距离，以促进更高效的导航。值得注意的是，SPF以闭环控制方式进行导航，使无人机能够在动态环境中跟踪动态目标。SPF在DRL模拟基准上创下了新的最先进水平，以63%的绝对优势超过了之前的最佳方法。在广泛的真实世界评估中，SPF以较大优势超过了强基线。我们还进行了全面的消融研究，以突出我们设计选择的有效性。最后，SPF展示了在不同VLM上的显著泛化能力。项目页面：https://spf-web.pages.dev
英文摘要：
We present See, Point, Fly (SPF), a training-free aerial vision-and-language navigation (AVLN) framework built atop vision-language models (VLMs). SPF is capable of navigating to any goal based on any type of free-form instructions in any kind of environment. In contrast to existing VLM-based approaches that treat action prediction as a text generation task, our key insight is to consider action prediction for AVLN as a 2D spatial grounding task. SPF harnesses VLMs to decompose vague language instructions into iterative annotation of 2D waypoints on the input image. Along with the predicted traveling distance, SPF transforms predicted 2D waypoints into 3D displacement vectors as action commands for UAVs. Moreover, SPF also adaptively adjusts the traveling distance to facilitate more efficient navigation. Notably, SPF performs navigation in a closed-loop control manner, enabling UAVs to follow dynamic targets in dynamic environments. SPF sets a new state of the art in DRL simulation benchmark, outperforming the previous best method by an absolute margin of 63%. In extensive real-world evaluations, SPF outperforms strong baselines by a large margin. We also conduct comprehensive ablation studies to highlight the effectiveness of our design choice. Lastly, SPF shows remarkable generalization to different VLMs. Project page: https://spf-web.pages.dev
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
