arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.08096cs.ROcs.AIcs.ETcs.LG
收藏

基于混合动作的多目标兼容自动驾驶强化学习

Hybrid Action Based Reinforcement Learning for Multi-Objective Compatible Autonomous Driving

College of Automotive and Energy Engineering, Tongji University(同济大学汽车与能源工程学院);Department of Data and Systems Engineering, University of Hong Kong(香港大学数据与系统工程系)

Guizhe Jin, Zhuoren Li, Bo Leng, Wei Han, Lu Xiong, Chen Sun

AI总结:

本文提出一种多目标集成-批评强化学习方法,通过混合参数化动作空间结构和不确定性探索机制,提升自动驾驶在多目标兼容性、效率和安全性方面的性能。

摘要:

强化学习(RL)在解决自动驾驶决策和控制问题中表现出色,但驾驶是多属性问题,导致当前RL方法在多目标兼容性方面面临挑战,特别是在策略更新和执行方面。本文提出一种多目标集成-批评强化学习方法,通过构建先进的MORL架构,利用独立奖励函数聚焦不同目标,并整合混合参数化动作空间结构,生成既包含抽象指导又包含具体控制命令的驾驶动作。此外,开发了一种基于不确定性的探索机制,以支持混合动作,从而更快速地学习多目标兼容的策略。实验结果表明,在基于模拟器和HighD数据集的多车道高速公路场景中,该方法在效率、动作一致性和安全性方面有效学习了多目标兼容的自动驾驶。

英文摘要:

Reinforcement Learning (RL) has shown excellent performance in solving decision-making and control problems of autonomous driving, which is increasingly applied in diverse driving scenarios. However, driving is a multi-attribute problem, leading to challenges in achieving multi-objective compatibility for current RL methods, especially in both policy updating and policy execution. On the one hand, a single value evaluation network limits the policy updating in complex scenarios with coupled driving objectives. On the other hand, the common single-type action space structure limits driving flexibility or results in large behavior fluctuations during policy execution. To this end, we propose a Multi-objective Ensemble-Critic reinforcement learning method with Hybrid Parametrized Action for multi-objective compatible autonomous driving. Specifically, an advanced MORL architecture is constructed, in which the ensemble-critic focuses on different objectives through independent reward functions. The architecture integrates a hybrid parameterized action space structure, and the generated driving actions contain both abstract guidance that matches the hybrid road modality and concrete control commands. Additionally, an uncertainty-based exploration mechanism that supports hybrid actions is developed to learn multi-objective compatible policies more quickly. Experimental results demonstrate that, in both simulator-based and HighD dataset-based multi-lane highway scenarios, our method efficiently learns multi-objective compatible autonomous driving with respect to efficiency, action consistency, and safety.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑