arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2412.09647cs.ROcs.CVcs.LG
收藏

Bench2Drive-R:通过生成式模型将真实世界数据转化为反应式闭环自动驾驶基准

Bench2Drive-R: Turning Real World Data into Reactive Closed-Loop Autonomous Driving Benchmark by Generative Model

Shanghai Jiao Tong University(上海交通大学)

Junqi You, Xiaosong Jia, Zhiyuan Zhang, Yutao Zhu, Junchi Yan

AI总结:

针对端到端自动驾驶现有闭环评估真实感不足或智能体无反应性的问题,提出解耦渲染与行为推演的Bench2Drive-R生成框架,通过时空一致性设计实现反应式评估,性能达SOTA并将集成至nuPlan开源。

摘要:

对于端到端自动驾驶(E2E-AD)而言,评估体系仍是一个未解决的问题。现有闭环评估协议通常依赖CARLA等模拟器,真实感不足;而使用真实世界视觉数据的NAVSIM,却受限于短时域内的固定规划轨迹,且假设其他智能体不具备反应性。\n 我们提出Bench2Drive-R,这是一个支持反应式闭环评估的生成式框架。与现有面向自动驾驶的视频生成模型不同,所提设计专为交互式仿真定制:通过应用独立的行为控制器来模拟周围智能体的反应,实现传感器渲染与行为推演的解耦。由此,渲染器可专注于图像保真度、控制依从性和时空一致性。在时间一致性方面,由于仿真具有逐步交互的特性,我们设计了带高斯模糊的噪声调制时间编码器,以促进图像序列的长时域自回归推演,同时避免分布偏移恶化。在空间一致性方面,引入了以空间最近邻图像为参考的检索机制,确保生成过程中的场景级渲染保真度。通过3D相对位置编码显式建模目标与参考之间的空间关系,并通过分层采样和无分类器引导缓解对参考图像的潜在过度依赖。\n 我们将Bench2Drive-R的生成质量与现有生成模型进行了对比,取得了当前最优(state-of-the-art)性能。我们进一步将Bench2Drive-R集成到nuPlan中,并结合闭环仿真结果评估生成质量。我们将开源代码。

英文摘要:

For end-to-end autonomous driving (E2E-AD), the evaluation system remains an open problem. Existing closed-loop evaluation protocols usually rely on simulators like CARLA being less realistic; while NAVSIM using real-world vision data, yet is limited to fixed planning trajectories in short horizon and assumes other agents are not reactive. We introduce Bench2Drive-R, a generative framework that enables reactive closed-loop evaluation. Unlike existing video generative models for AD, the proposed designs are tailored for interactive simulation, where sensor rendering and behavior rollout are decoupled by applying a separate behavioral controller to simulate the reactions of surrounding agents. As a result, the renderer could focus on image fidelity, control adherence, and spatial-temporal coherence. For temporal consistency, due to the step-wise interaction nature of simulation, we design a noise modulating temporal encoder with Gaussian blurring to encourage long-horizon autoregressive rollout of image sequences without deteriorating distribution shifts. For spatial consistency, a retrieval mechanism, which takes the spatially nearest images as references, is introduced to to ensure scene-level rendering fidelity during the generation process. The spatial relations between target and reference are explicitly modeled with 3D relative position encodings and the potential over-reliance of reference images is mitigated with hierarchical sampling and classifier-free guidance. We compare the generation quality of Bench2Drive-R with existing generative models and achieve state-of-the-art performance. We further integrate Bench2Drive-R into nuPlan and evaluate the generative qualities with closed-loop simulation results. We will open source our code.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑