系统化多智能体视觉与语言导航：形式化、基准与方法
Systematic Multi-Agent Vision-and-Language Navigation: Formulation, Benchmark, and Method
浏览论文内容
中文总结 AI 辅助
本文首次系统形式化多智能体视觉与语言导航为受约束协调问题，构建含11,724情节的MAVLN基准，并提出TRISS系统，实验揭示协调挑战与改进空间。
中文摘要 AI 辅助
视觉与语言导航（VLN）长期以来主要关注单个智能体遵循单条指令，然而许多现实应用需要机器人团队协作完成超出任何单个智能体能力的任务。我们提出了系统化多智能体视觉与语言导航，据我们所知，这是首次将多智能体VLN系统化地形式化为一个受约束的协调问题：每个任务由携带依赖和资源约束（存在锁与持有链）的子任务组成。一个经过验证的四阶段构建流水线将该任务实例化为MAVLN，包含145个场景中的11,724个情节，智能体团队最多可达四个，并涵盖三种指令模式，同时配套了定制化的约束感知指标。我们进一步提出了TRISS，一个具备协调能力的导航系统，它结合了基于大语言模型的子任务调度器、将每个智能体的探索转化为团队知识的共享拓扑记忆，以及一种冲突感知的执行机制，可将同时产生的意图实现为无碰撞路径。大量实验将TRISS确立为综合基线，并揭示了在调度、规划与执行方面仍有显著改进空间，突显了在MAVLN任务约束下进行协调的挑战。项目页面：此https URL。
英文摘要
Vision-and-Language Navigation (VLN) has largely focused on a single agent following a single instruction, yet many real-world applications require teams of robots to tackle tasks beyond the capabilities of any individual agent. We present Systematic Multi-Agent Vision-and-Language Navigation, providing, to our knowledge, the first systematic formalization of multi-agent VLN as a constrained coordination problem: each mission consists of subtasks carrying dependency and resource constraints (presence locks and holding chains). A verified four-stage crafting pipeline instantiates the task as MAVLN, comprising 11,724 episodes across 145 scenes with teams of up to four agents under three instruction regimes, accompanied by tailored constraint-aware metrics. We further present TRISS, a coordination-ready navigation system coupling an LLM-based subtask scheduler, a shared topological memory that turns each agent's exploration into team knowledge, and a conflict-aware execution mechanism that realizes simultaneous intentions as collision-free routes. Extensive experiments establish TRISS as a comprehensive baseline and reveal substantial room for improvement across scheduling, planning, and execution, highlighting the challenges of coordinating under MAVLN task constraints. Project page: https://xyz9911.github.io/mavln.
发表机构
- Shanghai Jiao Tong University（上海交通大学）
机构由 AI 辅助整理，请以论文原文为准。
