arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2610.20121cs.RO
收藏

HM3D-M3:多智能体、多模态、多目标目标导航基准

HM3D-M3: A Benchmark for Multi-Agent, Multi-Modal, Multi-Object Goal Navigation

Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);School of Computer Science, Nanjing University(南京大学计算机科学学院);School of Intelligence Science and Technology, Nanjing University(南京大学智能科学与技术学院);School of Electrical and Automation Engineering, Nanjing Normal University(南京师范大学电气与自动化工程学院)

Hongyu Ding, Kanyu Xu, Mingjia Chen, Zixuan Chen, Jieqi Shi, Jing Huo, Yifan Zhang, Yang Gao

AI总结:

针对现有基准缺失多智能体、多模态、多目标目标导航的问题,提出HM3D-M3基准及两个基线方法,验证其性能优于适配方法与零样本方法,还部署了异构三机器人团队。

摘要:

人工智能智能体与机器人技术的快速进步,使得由机器人团队为人类提供服务成为不久的将来的事。这类机器人团队会同时接收多个目标,每个目标以人类方便的形式指定,比如类别名称、照片或口语描述。现有基准分别研究多智能体、多模态和多目标目标导航,缺少将三者置于同一任务场景的基准。为填补这一空白,我们提出HM3D-M3:K个智能体组成的团队从 photorealistic 室内场景的分散位置出发,搜寻M个无序的目标对象,每个目标以开放词汇类别、实例图像或实例描述的形式给出。HM3D-M3提供了379个开放词汇类别下3926个目标对应的1000个验证任务场景、10万个观测-分配训练样本,以及两个团队级指标:目标级成功率(SR)和归一化完成时间的SPL。我们还贡献了两个基线方法:HiGA是在冻结的开放词汇导航策略之上训练的分层目标分配器,LaViRA-M3是无需训练的VLM智能体团队,具备共享建图与进度分析能力。适配的多机器人及零样本方法达到16%-27%的SR,而我们的基线方法达到38%-41%,各有互补优势与不同失败模式。此外,我们将LaViRA-M3部署在异构三机器人团队上。项目页面:this https URL

英文摘要:

Rapid progress in AI agents and robotics makes it a matter of the near future that humans are served by a team of robots. Such a team will receive several goals at once, each specified in whatever form is convenient for the human, such as a category name, a photo, or a spoken description. Existing benchmarks study multi-agent, multi-modal, and multi-object goal navigation in isolation, and a benchmark that places all three in one episode is missing. To fill this gap, we introduce HM3D-M3. A team of K agents starts at distributed locations in a photorealistic indoor scene and searches for an unordered set of M object goals, each given as an open-vocabulary category, an instance image, or an instance description. HM3D-M3 provides 1,000 validation episodes over 3,926 goals from 379 open-vocabulary categories, 100K observation-assignment training samples, and two team-level metrics, goal-level SR and makespan-normalized SPL. We further contribute two baselines. HiGA is a trained hierarchical goal assigner on top of a frozen open-vocabulary navigation policy, and LaViRA-M3 is a training-free team of VLM agents with shared mapping and progress analysis. Adapted multi-robot and zero-shot methods reach 16--27% SR, while our baselines reach 38--41% with complementary strengths and distinct failure modes. Furthermore, we deploy LaViRA-M3 on a heterogeneous three-robot team. Project page: https://robo-lavira.github.io/hm3d-m3/

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑