arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2512.10046cs.AI
收藏

SimWorld-Robotics: 为多模态机器人导航与协作合成逼真动态城市环境

SimWorld-Robotics: Synthesizing Photorealistic and Dynamic Urban Environments for Multimodal Robot Navigation and Collaboration

University of Virginia(弗吉尼亚大学);UC San Diego(加州大学圣地亚哥分校);Johns Hopkins University(约翰霍普金斯大学);Carnegie Mellon University(卡内基梅隆大学);University of Michigan(密歇根大学)

Yan Zhuang, Jiawei Ren, Xiaokang Ye, Jianzhi Shen, Ruixuan Zhang, Tianai Yue, Muhammad Faayez, Xuhong He, Ziqiao Ma, Lianhui Qin, Zhiting Hu, Tianmin Shu

AI总结:

SimWorld-Robotics通过合成逼真动态城市环境,提出两个多模态机器人基准测试,评估机器人在复杂场景中的导航、协作与通信能力。

摘要:

近期基础模型的进步在开发能够在一个开放的场景中根据多模态输入执行多样化任务的通用机器人方面显示出有前途的结果。然而,当前的研究主要集中在室内、家庭场景上。在本工作中,我们提出了SimWorld-Robotics(SWR),一个用于大规模、逼真城市环境的具身人工智能仿真平台。该平台基于Unreal Engine 5,能够程序生成无限的逼真城市场景,其中包含动态元素,如行人和交通系统,从而在真实感、复杂性和可扩展性方面超越了先前的城市仿真。它还支持多机器人控制和通信。借助这些关键功能,我们构建了两个具有挑战性的机器人基准测试:(1)一个多模态指令跟随任务,其中机器人必须在行人和交通存在的条件下,根据视觉-语言导航指令到达目的地;(2)一个多智能体搜索任务,其中两个机器人必须通信以协作定位并相遇。与现有基准测试不同,这两个新基准测试全面评估了现实场景中广泛的机器人能力,包括(1)多模态指令定位,(2)在大规模环境中的3D空间推理,(3)安全、长距离导航与行人和交通,(4)多机器人协作,以及(5)基于现实的通信。我们的实验结果表明,最先进的模型,包括视觉-语言模型(VLMs),在我们的任务上表现不佳,缺乏在城市环境中所需的稳健感知、推理和规划能力。

英文摘要:

Recent advances in foundation models have shown promising results in developing generalist robotics that can perform diverse tasks in open-ended scenarios given multimodal inputs. However, current work has been mainly focused on indoor, household scenarios. In this work, we present SimWorld-Robotics~(SWR), a simulation platform for embodied AI in large-scale, photorealistic urban environments. Built on Unreal Engine 5, SWR procedurally generates unlimited photorealistic urban scenes populated with dynamic elements such as pedestrians and traffic systems, surpassing prior urban simulations in realism, complexity, and scalability. It also supports multi-robot control and communication. With these key features, we build two challenging robot benchmarks: (1) a multimodal instruction-following task, where a robot must follow vision-language navigation instructions to reach a destination in the presence of pedestrians and traffic; and (2) a multi-agent search task, where two robots must communicate to cooperatively locate and meet each other. Unlike existing benchmarks, these two new benchmarks comprehensively evaluate a wide range of critical robot capacities in realistic scenarios, including (1) multimodal instructions grounding, (2) 3D spatial reasoning in large environments, (3) safe, long-range navigation with people and traffic, (4) multi-robot collaboration, and (5) grounded communication. Our experimental results demonstrate that state-of-the-art models, including vision-language models (VLMs), struggle with our tasks, lacking robust perception, reasoning, and planning abilities necessary for urban environments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑