基于变换器的多智能体强化学习用于结构化和非结构化空域中的分离保障
Transformer-based Multi-agent Reinforcement Learning for Separation Assurance in Structured and Unstructured Airspaces
AI总结：
本文提出基于变换器的多智能体强化学习方法，通过相对极坐标状态空间和改进的编码器架构，实现结构化和非结构化空域中的高效飞机分离保障。
摘要：
传统的基于优化的流量控制依赖于严格遵守预计算的调度，这限制了高级空域机动（AAM）随机操作所需的灵活性。相比之下，多智能体强化学习（MARL）提供了一种去中心化、自适应的框架，能够更好地处理不确定性，这对于安全飞机分离保障至关重要。尽管这一优势，当前的MARL方法往往过度适应特定的空域结构，限制了它们对新配置的适应性。为了提高泛化能力，我们将MARL问题重新表述为相对极坐标状态空间，并在多样化的交通模式和交叉角下训练一个变换器编码器模型。学习的模型提供速度建议以解决冲突，同时保持飞机接近其期望的巡航速度。在我们的实验中，我们在结构化和非结构化空域中评估了编码器深度为1、2和3层的情况，并发现单个编码器配置优于更深层的变体，产生接近零的近空域碰撞率和比更深层配置更短的分离丧失侵入时间。此外，我们还展示了相同的配置优于仅使用注意力机制设计的基线模型。总的来说，我们的结果表明，新制定的状态表示、神经网络架构的新设计和提出的训练策略提供了一种适应性强且可扩展的去中心化解决方案，用于结构化和非结构化空域中的飞机分离保障。
英文摘要：
Conventional optimization-based metering depends on strict adherence to precomputed schedules, which limits the flexibility required for the stochastic operations of Advanced Air Mobility (AAM). In contrast, multi-agent reinforcement learning (MARL) offers a decentralized, adaptive framework that can better handle uncertainty, required for safe aircraft separation assurance. Despite this advantage, current MARL approaches often overfit to specific airspace structures, limiting their adaptability to new configurations. To improve generalization, we recast the MARL problem in a relative polar state space and train a transformer encoder model across diverse traffic patterns and intersection angles. The learned model provides speed advisories to resolve conflicts while maintaining aircraft near their desired cruising speeds. In our experiments, we evaluated encoder depths of 1, 2, and 3 layers in both structured and unstructured airspaces, and found that a single encoder configuration outperformed deeper variants, yielding near-zero near mid-air collision rates and shorter loss-of-separation infringements than the deeper configurations. Additionally, we showed that the same configuration outperforms a baseline model designed purely with attention. Together, our results suggest that the newly formulated state representation, novel design of neural network architecture, and proposed training strategy provide an adaptable and scalable decentralized solution for aircraft separation assurance in both structured and unstructured airspaces.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
