arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.02613cs.LGcs.RO
收藏

通过拉格朗日引导的流匹配实现实时生成策略用于自动驾驶

Real-Time Generative Policy via Langevin-Guided Flow Matching for Autonomous Driving

School of Vehicle and Mobility, Tsinghua University(车辆与移动系统学院,清华大学);College of Artificial Intelligence, Tsinghua University(人工智能学院,清华大学)

Tianze Zhu, Yinuo Wang, Wenjun Zou, Tianyi Zhang, Likun Wang, Letian Tao, Feihong Zhang, Yao Lyu, Shengbo Eben Li

AI总结:

DACER-F通过流匹配实现实时生成策略,提升自动驾驶中的决策效率与性能。

摘要:

强化学习(RL)是自动驾驶系统中的基本方法,其中生成性策略通过利用其建模复杂分布的能力来增强探索具有相当大的潜力。然而,其固有的高推理延迟严重阻碍了其在实时决策和控制中的部署。为了解决这个问题,我们通过将流匹配引入在线RL中,提出了一种扩散actor-critic带有熵调节器(DACER-F),从而能够在单步推理中生成具有竞争力的动作。通过利用拉格朗日动力学和Q函数的梯度,DACER-F动态地从经验回放中优化动作,朝着一个平衡高Q值信息与探索行为的目标分布。然后,流策略被训练以高效地学习从简单的先验分布到此动态目标的映射。在复杂的多车道和交叉口模拟中,DACER-F优于基准扩散actor-critic带有熵调节器(DACER)和分布性软actor-critic(DSAC),同时保持超低的推理延迟。DACER-F进一步在标准RL基准DeepMind Control Suite(DMC)上展示了其可扩展性,实现了人体站立任务的775.8分,超过了先前方法。这些结果共同确立了DACER-F作为一种高性能且计算高效的RL算法。

英文摘要:

Reinforcement learning (RL) is a fundamental methodology in autonomous driving systems, where generative policies exhibit considerable potential by leveraging their ability to model complex distributions to enhance exploration. However, their inherent high inference latency severely impedes their deployment in real-time decision-making and control. To address this issue, we propose diffusion actor-critic with entropy regulator via flow matching (DACER-F) by introducing flow matching into online RL, enabling the generation of competitive actions in a single inference step. By leveraging Langevin dynamics and gradients of the Q-function, DACER-F dynamically optimizes actions from experience replay toward a target distribution that balances high Q-value information with exploratory behavior. The flow policy is then trained to efficiently learn a mapping from a simple prior distribution to this dynamic target. In complex multi-lane and intersection simulations, DACER-F outperforms baselines diffusion actor-critic with entropy regulator (DACER) and distributional soft actor-critic (DSAC), while maintaining an ultra-low inference latency. DACER-F further demonstrates its scalability on standard RL benchmark DeepMind Control Suite (DMC), achieving a score of 775.8 in the humanoid-stand task and surpassing prior methods. Collectively, these results establish DACER-F as a high-performance and computationally efficient RL algorithm.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑