arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2003.04371cs.AIcs.LGcs.SYeess.SY
收藏

一种用于互联自主车辆安全高效行为规划的多智能体强化学习方法

A Multi-Agent Reinforcement Learning Approach For Safe and Efficient Behavior Planning Of Connected Autonomous Vehicles

University of Connecticut(康涅狄格大学);University of Florida(佛罗里达大学)

Songyang Han, Shanglin Zhou, Jiangwei Wang, Lynn Pepin, Caiwen Ding, Jie Fu, Fei Miao

摘要:

无线技术的最新进展使联网的自动驾驶车辆(CAV)能够通过车辆到车辆(V2 V)通信收集有关其环境的信息。在这项工作中,我们设计了一个基于信息共享的多智能体强化学习(MARL)框架的CAV,利用额外的信息时,作出决策,以提高交通效率和安全。我们提出的安全Actor-Critic算法有两个新的技术:截断Q函数和安全动作映射。截断的Q函数利用来自相邻CAV的共享信息,使得Q函数的联合状态和动作空间在我们的大规模CAV系统的算法中不会增长。证明了截断Q函数与整体Q函数的逼近误差的界。安全动作映射为基于控制屏障函数的训练和执行提供了可证明的安全保证。使用CARLA模拟器进行实验,我们表明,我们的方法可以提高CAV系统的效率,平均速度和舒适度在不同的CAV比率和不同的交通密度。我们还表明,我们的方法避免了执行不安全的行动,并始终保持与其他车辆的安全距离。我们构建了一个弯道障碍场景,以表明共享视觉可以帮助CAV更早地观察障碍物并采取行动以避免交通堵塞。

英文摘要:

The recent advancements in wireless technology enable connected autonomous vehicles (CAVs) to gather information about their environment by vehicle-to-vehicle (V2V) communication. In this work, we design an information-sharing-based multi-agent reinforcement learning (MARL) framework for CAVs, to take advantage of the extra information when making decisions to improve traffic efficiency and safety. The safe actor-critic algorithm we propose has two new techniques: the truncated Q-function and safe action mapping. The truncated Q-function utilizes the shared information from neighboring CAVs such that the joint state and action spaces of the Q-function do not grow in our algorithm for a large-scale CAV system. We prove the bound of the approximation error between the truncated-Q and global Q-functions. The safe action mapping provides a provable safety guarantee for both the training and execution based on control barrier functions. Using the CARLA simulator for experiments, we show that our approach can improve the CAV system's efficiency in terms of average velocity and comfort under different CAV ratios and different traffic densities. We also show that our approach avoids the execution of unsafe actions and always maintains a safe distance from other vehicles. We construct an obstacle-at-corner scenario to show that the shared vision can help CAVs to observe obstacles earlier and take action to avoid traffic jams.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑