arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.14908cs.ROcs.CV
收藏

PerlAD:基于伪模拟的强化学习在闭环端到端自动驾驶中的应用

PerlAD: Towards Enhanced Closed-loop End-to-end Autonomous Driving with Pseudo-simulation-based Reinforcement Learning

School of Automation and Electrical Engineering, University of Science and Technology Beijing(北京科技大学自动化与电气工程学院);Xiaomi EV(小牛电动车);State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所多模态人工智能系统国家重点实验室)

Yinfeng Gao, Qichao Zhang, Deqing Liu, Zhongpu Xia, Guang Li, Kun Ma, Guang Chen, Hangjun Ye, Long Chen, Da-Wei Ding, Dongbin Zhao

AI总结:

PerlAD通过伪模拟强化学习方法解决闭环端到端自动驾驶中训练与现实需求不匹配的问题,利用向量空间构建伪模拟环境,结合预测世界模型和分层解耦规划器,实现高效训练和规划,实验表明其在Bench2Drive和DOS基准上均表现优异。

摘要:

基于模仿学习的端到端自动驾驶策略在闭环执行中常因开放环训练目标与真实驾驶需求不匹配而遇到困难。虽然强化学习通过奖励信号直接优化驾驶目标,但基于渲染的训练环境引入了渲染间隙且计算成本高。为克服这些挑战,我们提出了一种基于伪模拟的强化学习方法PerlAD,利用离线数据集构建向量空间伪模拟,实现高效无渲染的试错训练。为弥合静态数据集与动态闭环环境之间的差距,PerlAD引入预测世界模型,根据自动驾驶车辆的计划生成反应性代理轨迹。此外,为促进高效规划,PerlAD采用分层解耦规划器,结合模仿学习生成横向路径和强化学习优化纵向速度。全面的实验结果表明,PerlAD在Bench2Drive基准上实现了最先进的性能,比先前的端到端强化学习方法在驾驶得分上提高了10.29%,无需昂贵的在线交互。此外,对DOS基准的进一步评估进一步证实了其在安全关键遮挡场景中的可靠性。

英文摘要:

End-to-end autonomous driving policies based on Imitation Learning (IL) often struggle in closed-loop execution due to the misalignment between inadequate open-loop training objectives and real driving requirements. While Reinforcement Learning (RL) offers a solution by directly optimizing driving goals via reward signals, the rendering-based training environments introduce the rendering gap and are inefficient due to high computational costs. To overcome these challenges, we present a novel Pseudo-simulation-based RL method for closed-loop end-to-end autonomous driving, PerlAD. Based on offline datasets, PerlAD constructs a pseudo-simulation that operates in vector space, enabling efficient, rendering-free trial-and-error training. To bridge the gap between static datasets and dynamic closed-loop environments, PerlAD introduces a prediction world model that generates reactive agent trajectories conditioned on the ego vehicle's plan. Furthermore, to facilitate efficient planning, PerlAD utilizes a hierarchical decoupled planner that combines IL for lateral path generation and RL for longitudinal speed optimization. Comprehensive experimental results demonstrate that PerlAD achieves state-of-the-art performance on the Bench2Drive benchmark, surpassing the previous E2E RL method by 10.29% in Driving Score without requiring expensive online interactions. Additional evaluations on the DOS benchmark further confirm its reliability in handling safety-critical occlusion scenarios.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑