Proper functioning of connected and automated vehicles (CAVs) is crucial for the safety and efficiency of future intelligent transport systems. Meanwhile, transitioning to fully autonomous driving requires a long period of mixed autonomy traffic, including both CAVs and human-driven vehicles. Thus, collaboration decision-making for CAVs is essential to generate appropriate driving behaviors to enhance the safety and efficiency of mixed autonomy traffic. In recent years, deep reinforcement learning (DRL) has been widely used in solving decision-making problems. However, the existing DRL-based methods have been mainly focused on solving the decision-making of a single CAV. Using the existing DRL-based methods in mixed autonomy traffic cannot accurately represent the mutual effects of vehicles and model dynamic traffic environments. To address these shortcomings, this article proposes a graph reinforcement learning (GRL) approach for multi-agent decision-making of CAVs in mixed autonomy traffic. First, a generic and modular GRL framework is designed. Then, a systematic review of DRL and GRL methods is presented, focusing on the problems addressed in recent research. Moreover, a comparative study on different GRL methods is further proposed based on the designed framework to verify the effectiveness of GRL methods. Results show that the GRL methods can well optimize the performance of multi-agent decision-making for CAVs in mixed autonomy traffic compared to the DRL methods. Finally, challenges and future research directions are summarized. This study can provide a valuable research reference for solving the multi-agent decision-making problems of CAVs in mixed autonomy traffic and can promote the implementation of GRL-based methods into intelligent transportation systems. The source code of our work can be found at https://github.com/Jacklinkk/Graph_CAVs.
Human Observation-Inspired Trajectory Prediction for Autonomous Driving in Mixed-Autonomy Traffic Environments
University of Electronic Science and Technology of China（电子科技大学）；University of Macau（澳门大学）；Macau University of Science and Technology（澳门科技大学）；Tsinghua University（清华大学）
Flow: A Modular Learning Framework for Mixed Autonomy Traffic
Laboratory for Information and Decision Systems, Massachusetts Institute of Technology（信息与决策实验室，麻省理工学院）；Institute of Data, Systems, and Society, Massachusetts Institute of Technology（数据、系统与社会研究所，麻省理工学院）；Department of Mechanical Engineering, University of California, Berkeley（机械工程系，加州大学伯克利分校）
Artificial Intelligence for Modeling and Simulation of Mixed Automated and Human Traffic
Department of Transport & Planning, Delft University of Technology（代尔夫特理工大学交通与规划系）；Department of Industrial and Systems Engineering, University of Michigan, Dearborn（密歇根大学迪尔伯恩分校工业与系统工程系）；Tandon School of Engineering, New York University（纽约大学Tandon工程学院）
Temporal Transfer Learning for Traffic Optimization with Coarse-grained Advisory Autonomy
Department of Civil and Environmental Engineering, Massachusetts Institute of Technology（麻省理工学院土木与环境工程系）；Laboratory for Information & Decision Systems, Massachusetts Institute of Technology（麻省理工学院信息与决策系统实验室）；Department of Mechanical and Automotive Engineering, Seoul National University of Science and Technology（首尔科学技术大学机械与汽车工程系）
SafeCast: Risk-Responsive Motion Forecasting for Autonomous Vehicles
State Key Laboratory of Internet of Things for Smart City, University of Macau（澳门大学智慧城市物联网国家重点实验室）；Department of Computer and Information Science, University of Macau（澳门大学计算机与信息科学系）；Department of Civil and Environmental Engineering, University of Macau（澳门大学土木工程与环境工程系）
Trust-Aware Embodied Bayesian Persuasion for Mixed-Autonomy
Electrical & Computer Engineering, University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校电气与计算机工程系）；Industrial & Enterprise Systems Engineering, University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校工业与企业系统工程系）
Coordination for Connected and Automated Vehicles at Non-signalized Intersections: A Value Decomposition-based Multiagent Deep Reinforcement Learning Approach
Institute of Electrical Engineering Chinese Academy of Sciences（中国科学院电工研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Tsinghua University（清华大学）
Ethics-Aware Safe Reinforcement Learning for Rare-Event Risk Control in Interactive Urban Driving
Chair of Econometrics and Statistics Corresponding author esp. in the Transport Sector（计量经济学与统计学教授职位（交通运输领域特别对应作者））；Center for Scalable Data Analytics and Artificial Intelligence (ScaDS.AI)（可扩展数据与人工智能研究中心（ScaDS.AI））
