自动驾驶的可编辑场景模拟：基于协作式LLM智能体
Editable Scene Simulation for Autonomous Driving via Collaborative LLM-Agents
AI总结：
本文提出ChatSim，利用LLM智能体协作框架和多相机神经辐射场及光照估计方法，实现通过自然语言命令和外部数字资产进行可编辑的照片级真实感3D驾驶场景模拟。
摘要：
场景模拟在自动驾驶领域因其生成定制化数据的巨大潜力而受到广泛关注。然而，现有的可编辑场景模拟方法在用户交互效率、多相机照片级真实感渲染以及外部数字资产集成方面存在局限。为应对这些挑战，本文提出ChatSim，这是首个通过自然语言命令结合外部数字资产实现可编辑照片级真实感3D驾驶场景模拟的系统。为支持高灵活性的命令编辑，ChatSim利用大语言模型（LLM）智能体协作框架。为生成照片级真实感结果，ChatSim采用一种新颖的多相机神经辐射场方法。此外，为释放大量高质量数字资产的潜力，ChatSim采用一种新颖的多相机光照估计方法，以实现场景一致的资产渲染。我们在Waymo Open Dataset上的实验表明，ChatSim能够处理复杂的语言命令并生成相应的照片级真实感场景视频。
英文摘要：
Scene simulation in autonomous driving has gained significant attention because of its huge potential for generating customized data. However, existing editable scene simulation approaches face limitations in terms of user interaction efficiency, multi-camera photo-realistic rendering and external digital assets integration. To address these challenges, this paper introduces ChatSim, the first system that enables editable photo-realistic 3D driving scene simulations via natural language commands with external digital assets. To enable editing with high command flexibility,~ChatSim leverages a large language model (LLM) agent collaboration framework. To generate photo-realistic outcomes, ChatSim employs a novel multi-camera neural radiance field method. Furthermore, to unleash the potential of extensive high-quality digital assets, ChatSim employs a novel multi-camera lighting estimation method to achieve scene-consistent assets' rendering. Our experiments on Waymo Open Dataset demonstrate that ChatSim can handle complex language commands and generate corresponding photo-realistic scene videos.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
