D-HAL：交叉口自主管理中多Agent交互的分布式层次对抗学习
D-HAL: Distributed Hierarchical Adversarial Learning for Multi-Agent Interaction in Autonomous Intersection Management
摘要：
自主交叉口管理（AIM）为互联自主车辆（CAV）提供了一种无信号交叉口调度范例。分布式学习方法已成为AIM研究的一个引人注目的分支。与集中式AIM相比，分布式AIM能够以更低的成本部署到CAV上;与基于规则和基于优化的方法相比，基于学习的方法能够更灵活地处理各种复杂的实时交叉场景。深度强化学习（DRL）是分布式学习中解决AIM问题的主流方法。然而，多Agent的大规模同时交互决策以及交互引起的环境快速变化对DRL提出了挑战，使其回报曲线振荡且难以收敛，最终导致安全性和计算效率的折衷。为此，我们提出了一个非RL学习框架，称为分布式层次对抗学习（D-HAL）。该框架包括一个Actor网络，它在每个步骤中生成每个CAV的动作。即时评估器评估参与者网络在当前步骤的交互性能，而最终评估器从一系列交互中对整体轨迹进行最终评估。在这个框架中，行为的长期结果不再以折扣奖励的方式激励行动者网络，而是通过设计带有歧视性标签的对抗性损失函数。在一个四向六车道的交叉口进行了评价，该模型在保证安全和减少行程时间方面优于几种最先进的方法。
英文摘要：
Autonomous Intersection Management (AIM) provides a signal-free intersection scheduling paradigm for Connected Autonomous Vehicles (CAVs). Distributed learning method has emerged as an attractive branch of AIM research. Compared with centralized AIM, distributed AIM can be deployed to CAVs at a lower cost, and compared with rule-based and optimization-based method, learning-based method can treat various complicated real-time intersection scenarios more flexibly. Deep reinforcement learning (DRL) is the mainstream approach in distributed learning to address AIM problems. However, the large-scale simultaneous interactive decision of multiple agents and the rapid changes of environment caused by interactions pose challenges for DRL, making its reward curve oscillating and hard to converge, and ultimately leading to a compromise in safety and computing efficiency. For this, we propose a non-RL learning framework, called Distributed Hierarchical Adversarial Learning (D-HAL). The framework includes an actor network that generates the actions of each CAV at each step. The immediate discriminator evaluates the interaction performance of the actor network at the current step, while the final discriminator makes the final evaluation of the overall trajectory from a series of interactions. In this framework, the long-term outcome of the behavior no longer motivates the actor network in terms of discounted rewards, but rather through a designed adversarial loss function with discriminative labels. The proposed model is evaluated at a four-way-six-lane intersection, and outperforms several state-of-the-art methods on ensuring safety and reducing travel time.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
