The intelligent control of the traffic signal is critical to the optimization of transportation systems. To achieve global optimal traffic efficiency in large-scale road networks, recent works have focused on coordination among intersections, which have shown promising results. However, existing studies paid more attention to observations sharing among intersections (both explicit and implicit) and did not care about the consequences after decisions. In this paper, we design a multiagent coordination framework based on Deep Reinforcement Learning methods for traffic signal control, defined as γ-Reward that includes both original γ-Reward and γ-Attention-Reward. Specifically, we propose the Spatial Differentiation method for coordination which uses the temporal-spatial information in the replay buffer to amend the reward of each action. A concise theoretical analysis that proves the proposed model can converge to Nash equilibrium is given. By extending the idea of Markov Chain to the dimension of space-time, this truly decentralized coordination mechanism replaces the graph attention method and realizes the decoupling of the road network, which is more scalable and more in line with practice. The simulation results show that the proposed model remains a state-of-the-art performance even not use a centralized setting. Code is available in https://github.com/Skylark0924/Gamma Reward.
Emergent Coordination in Multi-Agent Language Models
D’Amore-McKim School of Business（达莫-麦克金商学院）；Khoury College of Computer Sciences（科里学院计算机科学学院）；Network Science Institute（网络科学研究所）；Northeastern University（东北大学）
MARL-FWC: Optimal Coordination of Freeway Traffic Control Measures
Cyber-Physical Systems Lab（智能物理系统实验室）；Egypt-Japan University of Science and Technology (E-JUST)（埃及-日本科学技术大学）；Faculty of Engineering (Shoubra)（工程学院（舒卜拉））；Benha University（班纳大学）；Faculty of Engineering（工程学院）；Alexandria University（亚历山大大学）
DSIP: A Dynamic Coordination Planner for Signal-Free Intersections using Diffusion-Model-Based Multi-Agent Motion Planning
Global Institute of Future Technology, Shanghai Jiao Tong University（上海交通大学全球未来技术学院）；School of Automation and Intelligent Sensing, Shanghai Jiao Tong University（上海交通大学自动化与智能感知学院）；Department of Electrical Engineering, The University of Texas at Arlington（德克萨斯大学阿灵顿分校电气工程系）
Steering traffic in cities is a very complex task, since improving efficiency involves the coordination of many actors. Traditional approaches attempt to optimize traffic lights for a particular density and configuration of traffic. The disadvantage of this lies in the fact that traffic densities and configurations change constantly. Traffic seems to be an adaptation problem rather than an optimization problem. We propose a simple and feasible alternative, in which traffic lights self-organize to improve traffic flow. We use a multi-agent simulation to study three self-organizing methods, whic…
Stanford University（斯坦福大学）；Microsoft（微软）；MIT Lincoln Labs（麻省理工学院林肯实验室）
Multi-agent reinforcement learning (MARL) requires coordination to efficiently solve certain tasks. Fully centralized control is often infeasible in such domains due to the size of joint action spaces. Coordination graph based formalization allows reasoning about the joint action based on the structure of interactions. However, they often require domain expertise in their design. This paper introduces the deep implicit coordination graph (DICG) architecture for such scenarios. DICG consists of a module for inferring the dynamic coordination graph structure which is then used by a graph neural …
AIcrowd；ETH Zurich（苏黎世联邦理工学院）；FHNW（瑞士西北应用科学与艺术大学）；University of Southern California（南加州大学）；Monash University（莫纳什大学）；HSE University（高等经济大学）；JetBrains Research；Netcetera
The Flatland competition aimed at finding novel approaches to solve the vehicle re-scheduling problem (VRSP). The VRSP is concerned with scheduling trips in traffic networks and the re-scheduling of vehicles when disruptions occur, for example the breakdown of a vehicle. While solving the VRSP in various settings has been an active area in operations research (OR) for decades, the ever-growing complexity of modern railway networks makes dynamic real-time scheduling of traffic virtually impossible. Recently, multi-agent reinforcement learning (MARL) has successfully tackled challenging tasks wh…
Siemens Technology and Services Pvt. Ltd.（西门子技术服务私人有限公司）；International Institute of Information Technology Hyderabad（海得拉巴国际信息技术学院）；Indian Institute of Science Bengaluru（班加罗尔印度科学研究所）
We use Asynchronous Advantage Actor Critic (A3C) for implementing an AI agent in the controllers that optimize flow of traffic across a single intersection and then extend it to multiple intersections by considering a multi-agent setting. We explore three different methodologies to address the multi-agent problem - (1) use of asynchronous property of A3C to control multiple intersections using a single agent (2) utilise self/competitive play among independent agents across multiple intersections and (3) ingest a global reward function among agents to introduce cooperative behavior between inte…
Cooperative coordination at unsignalized road intersections, which aims to improve the driving safety and traffic throughput for connected and automated vehicles, has attracted increasing interests in recent years. However, most existing investigations either suffer from computational complexity or cannot harness the full potential of the road infrastructure. To this end, we first present a dedicated intersection coordination framework, where the involved vehicles hand over their control authorities and follow instructions from a centralized coordinator. Then a unified cooperative trajectory o…
Decentralized traffic management of autonomous drones
Eötvös University of Budapest（布达佩斯厄特沃什大学）；MTA-ELTE Statistical and Biological Physics Research Group（匈牙利科学院-厄特沃什大学统计与生物物理研究组）；CollMot Robotics LLC（CollMot机器人有限责任公司）
Decentralized Coordination of Autonomous Traffic Through Advanced Air Mobility Corridors
Department of Aeronautics and Astronautics（航空航天系）；AIAA Student Member（美国航空航天学会学生会员）；Associate Dean of Engineering（工程副院长）；William E. Leonhard (1940) Professor（威廉·E·莱昂哈德（1940）教授）；AIAA Fellow（美国航空航天学会会士）
Recent work in decentralized, schedule-driven traffic control has demonstrated the ability to improve the efficiency of traffic flow in complex urban road networks. In this approach, a scheduling agent is associated with each intersection. Each agent senses the traffic approaching its intersection and in real-time constructs a schedule that minimizes the cumulative wait time of vehicles approaching the intersection over the current look-ahead horizon. In order to achieve network level coordination in a scalable manner, scheduling agents communicate only with their direct neighbors. Each time a…
Emergency vehicle (EMV) service is a key function of cities and is exceedingly challenging due to urban traffic congestion. The main reason behind EMV service delay is the lack of communication and cooperation between vehicles blocking EMVs. In this paper, we study the improvement of EMV service under V2X connectivity. We consider the establishment of dynamic queue jump lanes (DQJLs) based on real-time coordination of connected vehicles in the presence of non-connected human-driven vehicles. We develop a novel Markov decision process formulation for the DQJL coordination strategies, which expl…
Coordination recognition and subtle pattern prediction of future trajectories play a significant role when modeling interactive behaviors of multiple agents. Due to the essential property of uncertainty in the future evolution, deterministic predictors are not sufficiently safe and robust. In order to tackle the task of probabilistic prediction for multiple, interactive entities, we propose a coordination and trajectory prediction system (CTPS), which has a hierarchical structure including a macro-level coordination recognition module and a micro-level subtle pattern prediction module which so…
HALO: Hierarchical Reinforcement Learning for Large-Scale Adaptive Traffic Signal Control
University of Exeter Department of Computer Science Exeter UK（埃克塞特大学计算机科学系埃克塞特英国）；University of Warwick Department of Computer Science Coventry UK（沃里克大学计算机科学系科文特里英国）；University of Exeter（埃克塞特大学）；University of Warwick（沃里克大学）
The steady increase in the number of vehicles operating on the highways continues to exacerbate congestion, accidents, energy consumption, and greenhouse gas emissions. Emerging mobility systems, e.g., connected and automated vehicles (CAVs), have the potential to directly address these issues and improve transportation network efficiency and safety. In this paper, we consider a highway merging scenario and propose a framework for coordinating CAVs such that stop-and-go driving is eliminated. We use a decentralized form of the actor-critic approach to deep reinforcement learning$-$multi-agent …
