$ \text{T}^3 $ OMVP：基于变换器的时间和团队强化学习城市多车追踪算法
$ \text{T}^3 $OMVP: A Transformer-based Time and Team Reinforcement Learning Scheme for Observation-constrained Multi-Vehicle Pursuit in Urban Area
摘要：
智能车联网（IoV）与人工智能（AI）的结合将有助于智能交通系统（ITS）中的车辆决策。多车追击游戏（MVP）是一种多车协同捕获移动目标的能力，正逐渐成为研究的热点。虽然在开放空间环境下的MVP领域已经取得了一些成果，但城市区域带来的复杂道路结构和有限的移动空间给MVP游戏的解决带来了挑战。本文定义了一个观测约束MVP（Observation-constrained MVP，OMVP）问题，并提出了一个基于转换器的时间和团队强化学习方案（$ \text{T}^3 $ OMVP）来解决该问题。首先，一个新的多车辆追逐模型的基础上，分散部分观察马尔可夫决策过程（Dec-POMDP）构造实例化这个问题。其次，通过引入和修改基于变换器的观测序列，重新定义了QMIX算法，使其能够适应复杂的道路结构、有限的运动空间和有限的观测量，并结合车辆的观测值控制车辆追逐目标。第三，构建了一个多交叉口的城市环境，以验证所提出的方案。大量的实验结果表明，所提出的$ \text{T}^3 $ OMVP方案取得了显着的改善相对于最先进的QMIX方法的9.66%~ 106.25%。代码可在https://github.com/pipihaiziguai/T3OMVP上获得。
英文摘要：
Smart Internet of Vehicles (IoVs) combined with Artificial Intelligence (AI) will contribute to vehicle decision-making in the Intelligent Transportation System (ITS). Multi-Vehicle Pursuit games (MVP), a multi-vehicle cooperative ability to capture mobile targets, is becoming a hot research topic gradually. Although there are some achievements in the field of MVP in the open space environment, the urban area brings complicated road structures and restricted moving spaces as challenges to the resolution of MVP games. We define an Observation-constrained MVP (OMVP) problem in this paper and propose a Transformer-based Time and Team Reinforcement Learning scheme ($ \text{T}^3 $OMVP) to address the problem. First, a new multi-vehicle pursuit model is constructed based on decentralized partially observed Markov decision processes (Dec-POMDP) to instantiate this problem. Second, by introducing and modifying the transformer-based observation sequence, QMIX is redefined to adapt to the complicated road structure, restricted moving spaces and constrained observations, so as to control vehicles to pursue the target combining the vehicle's observations. Third, a multi-intersection urban environment is built to verify the proposed scheme. Extensive experimental results demonstrate that the proposed $ \text{T}^3 $OMVP scheme achieves significant improvements relative to state-of-the-art QMIX approaches by 9.66%~106.25%. Code is available at https://github.com/pipihaiziguai/T3OMVP.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
