迈向真实的无人机视觉语言导航：平台、基准与方法
Towards Realistic UAV Vision-Language Navigation: Platform, Benchmark, and Methodology
AI总结：
针对无人机视觉语言导航（VLN）中照搬地面设置的局限，本文提出OpenUAV仿真平台、含约12k轨迹的数据集及UAV-Need-Help基准，并设计了基于MLLM的分层轨迹生成导航方法，显著优于基线但与人类水平仍有差距。
摘要：
开发能够基于语言指令和视觉信息导航至目标位置的智能体，即视觉语言导航（VLN），已引起广泛关注。大多数研究集中在地面智能体上，而基于无人机（UAV）的VLN仍相对缺乏探索。近期在无人机视觉语言导航方面的努力主要采用地面VLN设置，依赖预定义的离散动作空间，忽略了地面与空中环境之间智能体运动动态的固有差异以及导航任务的复杂性。为了解决这些差异和挑战，我们从平台、基准和方法三个角度提出解决方案。为了在VLN任务中实现真实的无人机轨迹仿真，我们提出了OpenUAV平台，该平台具有多样化的环境、真实的飞行控制和广泛的算法支持。我们进一步在该平台上构建了一个面向目标的VLN数据集，包含约12k条轨迹，作为首个专为真实无人机VLN任务设计的数据集。为了应对复杂空中环境带来的挑战，我们提出了一个名为UAV-Need-Help的助手引导无人机目标搜索基准，该基准提供不同级别的引导信息，以帮助无人机更好地完成真实的VLN任务。我们还提出了一种无人机导航LLM，给定多视角图像、任务描述和助手指令，利用MLLM的多模态理解能力来联合处理视觉和文本信息，并执行分层轨迹生成。我们方法的评估结果显著优于基线模型，但与人类操作员取得的结果之间仍存在相当大的差距，凸显了UAV-Need-Help任务所面临的挑战。
英文摘要：
Developing agents capable of navigating to a target location based on language instructions and visual information, known as vision-language navigation (VLN), has attracted widespread interest. Most research has focused on ground-based agents, while UAV-based VLN remains relatively underexplored. Recent efforts in UAV vision-language navigation predominantly adopt ground-based VLN settings, relying on predefined discrete action spaces and neglecting the inherent disparities in agent movement dynamics and the complexity of navigation tasks between ground and aerial environments. To address these disparities and challenges, we propose solutions from three perspectives: platform, benchmark, and methodology. To enable realistic UAV trajectory simulation in VLN tasks, we propose the OpenUAV platform, which features diverse environments, realistic flight control, and extensive algorithmic support. We further construct a target-oriented VLN dataset consisting of approximately 12k trajectories on this platform, serving as the first dataset specifically designed for realistic UAV VLN tasks. To tackle the challenges posed by complex aerial environments, we propose an assistant-guided UAV object search benchmark called UAV-Need-Help, which provides varying levels of guidance information to help UAVs better accomplish realistic VLN tasks. We also propose a UAV navigation LLM that, given multi-view images, task descriptions, and assistant instructions, leverages the multimodal understanding capabilities of the MLLM to jointly process visual and textual information, and performs hierarchical trajectory generation. The evaluation results of our method significantly outperform the baseline models, while there remains a considerable gap between our results and those achieved by human operators, underscoring the challenge presented by the UAV-Need-Help task.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
