在互联驾驶环境中使用深度强化学习进行交通整形和滞后缓解
Traffic Shaping and Hysteresis Mitigation Using Deep Reinforcement Learning in a Connected Driving Environment
摘要：
基于多代理深度强化学习的流量整形框架。所提出的框架提供了一个关键的优势，现有的拥塞管理策略，这是能够减轻滞后现象。与现有的拥堵管理策略，重点是故障预防，建议的框架是非常有效的故障形成后。所提出的框架假设共享信息的自动驾驶汽车之间的部分连接。该框架要求由一维纵向控制定义的基本自主水平。该框架主要使用集中式训练、集中式执行的多智能体深度强化学习方法构建，其中纵向控制由加速或减速命令的信号定义，然后由所有智能体统一执行。用于框架的训练和测试的模型基于众所周知的双深度Q-学习算法，该算法将交通流内的流的平均状态作为模型输入，并以加速或减速值的形式输出动作。我们证明了该模型的能力，塑造交通状态，减轻滞后的负面影响，甚至提高交通流量超过其原来的水平。本文还确定了所需的CAV的最小百分比，以成功地塑造交通均匀分布的CAV的假设下，在循环系统。这项工作中说明的框架不仅显示了强化学习在解决这些挑战方面的理论适用性，而且还提出了一种现实的解决方案，只需部分连接和连续监控系统的平均速度，这可以通过使用现成的传感器来实现，这些传感器可以测量与CAV合理接近的车辆速度。
英文摘要：
A multi-agent deep reinforcement learning-based framework for traffic shaping. The proposed framework offers a key advantage over existing congestion management strategies which is the ability to mitigate hysteresis phenomena. Unlike existing congestion management strategies that focus on breakdown prevention, the proposed framework is extremely effective after breakdown formation. The proposed framework assumes partial connectivity between the automated vehicles which share information. The framework requires a basic level of autonomy defined by one-dimensional longitudinal control. This framework is primarily built using a centralized training, centralized execution multi-agent deep reinforcement learning approach, where longitudinal control is defined by signals of acceleration or deceleration commands which are then executed by all agents uniformly. The model undertaken for training and testing of the framework is based on the well-known Double Deep Q-Learning algorithm which takes the average state of flow within the traffic stream as the model input and outputs actions in the form of acceleration or deceleration values. We demonstrate the ability of the model to shape the state of traffic, mitigate the negative effects of hysteresis, and even improve traffic flow beyond its original level. This paper also identifies the minimum percentage of CAVs required to successfully shape the traffic under an assumption of uniformly distributed CAVs within the loop system. The framework illustrated in this work doesnt just show the theoretical applicability of reinforcement learning to tackle such challenges but also proposes a realistic solution that only requires partial connectivity and continuous monitoring of the average speed of the system, which can be achieved using readily available sensors that measure the speeds of vehicles in reasonable proximity to the CAVs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
