arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2501.13072cs.ROcs.AI
收藏

AdaWM:基于自适应世界模型的自动驾驶规划

AdaWM: Adaptive World Model based Planning for Autonomous Driving

Bosch Research North America & Bosch Center for Artificial Intelligence (BCAI)(博世北美研究院及博世人工智能中心);University of California, Davis(加州大学戴维斯分校)

Hang Wang, Xin Ye, Feng Tao, Chenbin Pan, Abhirup Mallik, Burhaneddin Yaman, Liu Ren, Junshan Zhang

AI总结:

针对自动驾驶中基于世界模型的强化学习在预训练-微调时因分布偏移导致性能下降的问题,提出AdaWM方法,通过不匹配识别与对齐驱动微调,在CARLA任务上显著提升性能。

摘要:

基于世界模型的强化学习(RL)已成为自动驾驶领域一种有前景的方法,它学习一个潜在动力学模型,并利用该模型训练规划策略。为了加速学习过程,通常采用预训练-微调范式,即通过预训练模型和离线学习的策略来初始化在线强化学习。然而,在新任务的在线交互过程中,简单地进行此类初始化可能导致性能显著下降。为应对这一挑战,我们首先分析了性能下降的原因,并确定了两个主要根源:由于分布偏移导致的规划策略不匹配和动力学模型不匹配。我们进一步分析了这些因素在微调过程中对性能下降的影响,研究结果表明,微调策略的选择在缓解这些影响方面起着关键作用。随后,我们提出了AdaWM,一种基于自适应世界模型的规划方法,包含两个关键步骤:(a)不匹配识别,量化不匹配程度并为微调策略提供信息;(b)对齐驱动的微调,根据需要使用高效的低秩更新选择性更新策略或模型。在具有挑战性的CARLA驾驶任务上进行的大量实验表明,AdaWM显著改善了微调过程,使自动驾驶系统获得更稳健和高效的性能。

英文摘要:

World model based reinforcement learning (RL) has emerged as a promising approach for autonomous driving, which learns a latent dynamics model and uses it to train a planning policy. To speed up the learning process, the pretrain-finetune paradigm is often used, where online RL is initialized by a pretrained model and a policy learned offline. However, naively performing such initialization in RL may result in dramatic performance degradation during the online interactions in the new task. To tackle this challenge, we first analyze the performance degradation and identify two primary root causes therein: the mismatch of the planning policy and the mismatch of the dynamics model, due to distribution shift. We further analyze the effects of these factors on performance degradation during finetuning, and our findings reveal that the choice of finetuning strategies plays a pivotal role in mitigating these effects. We then introduce AdaWM, an Adaptive World Model based planning method, featuring two key steps: (a) mismatch identification, which quantifies the mismatches and informs the finetuning strategy, and (b) alignment-driven finetuning, which selectively updates either the policy or the model as needed using efficient low-rank updates. Extensive experiments on the challenging CARLA driving tasks demonstrate that AdaWM significantly improves the finetuning process, resulting in more robust and efficient performance in autonomous driving systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑