Since 2016 federated learning (FL) has been an evolving topic of discussion in the artificial intelligence (AI) research community. Applications of FL led to the development and study of federated reinforcement learning (FRL). Few works exist on the topic of FRL applied to autonomous vehicle (AV) platoons. In addition, most FRL works choose a single aggregation method (usually weight or gradient aggregation). We explore FRL's effectiveness as a means to improve AV platooning by designing and implementing an FRL framework atop a custom AV platoon environment. The application of FRL in AV platooning is studied under two scenarios: (1) Inter-platoon FRL (Inter-FRL) where FRL is applied to AVs across different platoons; (2) Intra-platoon FRL (Intra-FRL) where FRL is applied to AVs within a single platoon. Both Inter-FRL and Intra-FRL are applied to a custom AV platooning environment using both gradient and weight aggregation to observe the performance effects FRL can have on AV platoons relative to an AV platooning environment trained without FRL. It is concluded that Intra-FRL using weight aggregation (Intra-FRLWA) provides the best performance for controlling an AV platoon. In addition, we found that weight aggregation in FRL for AV platooning provides increases in performance relative to gradient aggregation. Finally, a performance analysis is conducted for Intra-FRLWA versus a platooning environment without FRL for platoons of length 3, 4 and 5 vehicles. It is concluded that Intra-FRLWA largely out-performs the platooning environment that is trained without FRL.
As a strategy to reduce travel delay and enhance energy efficiency, platooning of connected and autonomous vehicles (CAVs) at non-signalized intersections has become increasingly popular in academia. However, few studies have attempted to model the relation between the optimal platoon size and the traffic conditions around the intersection. To this end, this study proposes an adaptive platoon based autonomous intersection control model powered by deep reinforcement learning (DRL) technique. The model framework has following two levels: the first level adopts a First Come First Serve (FCFS) res…
The coordination of multiple autonomous vehicles into convoys or platoons is expected on our highways in the near future. However, before such platoons can be deployed, the new autonomous behaviors of the vehicles in these platoons must be certified. An appropriate representation for vehicle platooning is as a multi-agent system in which each agent captures the "autonomous decisions" carried out by each vehicle. In order to ensure that these autonomous decision-making agents in vehicle platoons never violate safety requirements, we use formal verification. However, as the formal verification t…
Multi-Agent Deep Reinforcement Learning for Distributed and Autonomous Platoon Coordination via Speed-regulation over Large-scale Transportation Networks
Shanghai Research Institute for Intelligent Autonomous Systems, Tongji University（同济大学上海智能自主系统研究院）；Tongji University（同济大学）
Platooning Connected, Autonomous, and Human-Driven Vehicles: A Deep Reinforcement Learning-based Approach
School of Systems Science, Beijing Jiaotong University, Beijing 100044, China（北京交通大学系统科学学院）；Faculty of Science and Engineering, University of Nottingham Ningbo China（诺丁汉大学宁波分校理学院）
Deep Reinforcement Learning (DRL) is regarded as a potential method for car-following control and has been mostly studied to support a single following vehicle. However, it is more challenging to learn a stable and efficient car-following policy when there are multiple following vehicles in a platoon, especially with unpredictable leading vehicle behavior. In this context, we adopt an integrated DRL and Dynamic Programming (DP) approach to learn autonomous platoon control policies, which embeds the Deep Deterministic Policy Gradient (DDPG) algorithm into a finite-horizon value iteration framew…
Platooning and coordination are two implementation strategies that are frequently proposed for traffic control of connected and autonomous vehicles (CAVs) at signal-free intersections instead of using conventional traffic signals. However, few studies have attempted to integrate both strategies to better facilitate the CAV control at signal-free intersections. To this end, this study proposes a hierarchical control model, named COOR-PLT, to coordinate adaptive CAV platoons at a signal-free intersection based on deep reinforcement learning (DRL). COOR-PLT has a two-layer framework. The first la…
In the last few years, researchers have applied machine learning strategies in the context of vehicular platoons to increase the safety and efficiency of cooperative transportation. Reinforcement Learning methods have been employed in the longitudinal spacing control of Cooperative Adaptive Cruise Control systems, but to date, none of those studies have addressed problems of disturbance rejection in such scenarios. Characteristics such as uncertain parameters in the model and external interferences may prevent agents from reaching null-spacing errors when traveling at cruising speed. On the ot…
Institute of Radiocommunications（无线电通信研究所）；Poznan University of Technology（波兹南理工大学）
Reliable wireless communication between the autonomously driving cars is one of the fundamental needs for guaranteeing passenger safety and comfort. However, when the number of communicating cars increases, the transmission quality may be significantly degraded due to too high occupancy radio of the used frequency band. In this paper, we concentrate on the autonomous vehicle-platooning use-case, where intra-platoon communication is done in the dynamically selected frequency band, other than nominally devoted for such purposes. The carrier selection is done in a flexible manner with the support…
Agentic Trust Coordination for Federated Learning through Adaptive Thresholding and Autonomous Decision Making in Sustainable and Resilient Industrial Networks
London South Bank University（伦敦南岸大学）；Instituto de Telecomunicações（电信研究所）
Huawei Technologies Canada（华为加拿大技术公司）；McGill University（麦吉尔大学）；San Diego State University（圣地亚哥州立大学）
We consider the problem of joint channel assignment and power allocation in underlaid cellular vehicular-to-everything (C-V2X) systems where multiple vehicle-to-network (V2N) uplinks share the time-frequency resources with multiple vehicle-to-vehicle (V2V) platoons that enable groups of connected and autonomous vehicles to travel closely together. Due to the nature of high user mobility in vehicular environment, traditional centralized optimization approach relying on global channel information might not be viable in C-V2X systems with large number of users. Utilizing a multi-agent reinforceme…
Webank（微众银行）；Hong Kong University of Science and Technology（香港科技大学）
Reinforcement learning (RL) is widely used in autonomous driving tasks and training RL models typically involves in a multi-step process: pre-training RL models on simulators, uploading the pre-trained model to real-life robots, and fine-tuning the weight parameters on robot vehicles. This sequential process is extremely time-consuming and more importantly, knowledge from the fine-tuned model stays local and can not be re-used or leveraged collaboratively. To tackle this problem, we present an online federated RL transfer process for real-time knowledge extraction where all the participant age…
Tarbiat Modares University（塔尔比亚特莫达雷斯大学）；Shahrood University of Technology（沙赫鲁德理工大学）；TU Braunschweig（布伦瑞克工业大学）
This paper investigates the problem of age of information (AoI) aware radio resource management for a platooning system. Multiple autonomous platoons exploit the cellular wireless vehicle-to-everything (C-V2X) communication technology to disseminate the cooperative awareness messages (CAMs) to their followers while ensuring timely delivery of safety-critical messages to the Road-Side Unit (RSU). Due to the challenges of dynamic channel conditions, centralized resource management schemes that require global information are inefficient and lead to large signaling overheads. Hence, we exploit a d…
Semantic-Aware Resource Management for C-V2X Platooning via Multi-Agent Reinforcement Learning
School of Internet of Things Engineering, Jiangnan University（江南大学物联网工程学院）；Department of Electronic Engineering, State Key Laboratory of Space Network and Communications, and the Beijing National Research Center for Information Science and Technology, Tsinghua University（清华大学电子工程系、空间网络与通信国家重点实验室及北京信息科学与技术国家研究中心）；Department of Computer Science, Brunel University（布鲁内尔大学计算机科学系）；State Key Laboratory of ISN and the School of Telecommunications Engineering, Xidian University（西安电子科技大学信息与通信系统国家重点实验室及电信工程学院）；Department of Electronic Engineering, Shanghai Jiao Tong University（上海交通大学电子工程系）；Department of Electrical and Computer Engineering, the Hong Kong University of Science and Technology（香港科技大学电子与计算机工程系）
Securing Autonomous Vehicle Systems via Twin-Aware Federated Reinforcement Learning
North Carolina State University（北卡罗来纳州立大学）；University of Louisville（路易斯维尔大学）；University of Maryland（马里兰大学）；University of North Texas（北德克萨斯大学）；Wayne State University（韦恩州立大学）
Virginia Tech（弗吉尼亚理工大学）；Princeton University（普林斯顿大学）；Shenzhen Research Institute of Big Data（深圳大数据研究院）；Future Network of Intelligence Institute（未来网络智能研究院）；Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；University of Oulu（奥卢大学）
A new federated learning (FL) framework enabled by large-scale wireless connectivity is proposed for designing the autonomous controller of connected and autonomous vehicles (CAVs). In this framework, the learning models used by the controllers are collaboratively trained among a group of CAVs. To capture the varying CAV participation in the FL training process and the diverse local data quality among CAVs, a novel dynamic federated proximal (DFP) algorithm is proposed that accounts for the mobility of CAVs, the wireless fading channels, as well as the unbalanced and nonindependent and identic…
Nanyang Technological University (NTU)（南洋理工大学）；University of Maryland（马里兰大学）；China-Singapore International Joint Research Institute（中新国际联合研究院）
Object detection with on-board sensors (e.g., lidar, radar, and camera) play a crucial role in autonomous driving (AD), and these sensors complement each other in modalities. While crowdsensing may potentially exploit these sensors (of huge quantity) to derive more comprehensive knowledge, \textit{federated learning} (FL) appears to be the necessary tool to reach this potential: it enables autonomous vehicles (AVs) to train machine learning models without explicitly sharing raw sensory data. However, the multimodal sensors introduce various data heterogeneity across distributed AVs (e.g., labe…
Autonomous driving is an active research topic in both academia and industry. However, most of the existing solutions focus on improving the accuracy by training learnable models with centralized large-scale data. Therefore, these methods do not take into account the user's privacy. In this paper, we present a new approach to learn autonomous driving policy while respecting privacy concerns. We propose a peer-to-peer Deep Federated Learning (DFL) approach to train deep architectures in a fully decentralized manner and remove the need for central orchestration. We design a new Federated Autonom…
Electronic health record (EHR) data is collected by individual institutions and often stored across locations in silos. Getting access to these data is difficult and slow due to security, privacy, regulatory, and operational issues. We show, using ICU data from 58 different hospitals, that machine learning models to predict patient mortality can be trained efficiently without moving health data out of their silos using a distributed machine learning strategy. We propose a new method, called Federated-Autonomous Deep Learning (FADL) that trains part of the model using all data sources in a dist…
Communication Resources Constrained Hierarchical Federated Learning for End-to-End Autonomous Driving
The University of Hong Kong（香港大学）；Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Shenzhen Research Institute of Big Data（深圳大数据研究院）；Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ)（广东省人工智能与数字经济实验室（深圳））；Damo Institute, Alibaba（阿里巴巴达摩院）；the University of New South Wales（新南威尔士大学）
Poisoning Attacks on Federated Learning for Autonomous Driving
Umeå University（于默奥大学）；AI Sweden（瑞典人工智能研究院）；Royal Institute of Technology（皇家理工学院）；Chalmers University of Technology（查尔姆斯理工大学）；Dakota State University（达科他州立大学）；Scaleout Systems（Scaleout系统公司）
FedRobo: Federated Learning Driven Autonomous Inter Robots Communication For Optimal Chemical Sprays
University of Asia Pacific（亚太大学）；University of North Texas（北得克萨斯大学）；Green University of Bangladesh（孟加拉国绿色大学）；Southern Illinois University Carbondale（南伊利诺伊大学卡本代尔分校）
Northeastern University（东北大学）；University of Connecticut（康涅狄格大学）
Deep learning is the method of choice for trajectory prediction for autonomous vehicles. Unfortunately, its data-hungry nature implicitly requires the availability of sufficiently rich and high-quality centralized datasets, which easily leads to privacy leakage. Besides, uncertainty-awareness becomes increasingly important for safety-crucial cyber physical systems whose prediction module heavily relies on machine learning tools. In this paper, we relax the data collection requirement and enhance uncertainty-awareness by using Federated Learning on Connected Autonomous Vehicles with an uncertai…
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；University of New South Wales（新南威尔士大学）；Weizmann Institute of Science（魏茨曼科学研究所）；Princeton University（普林斯顿大学）；Southern University of Science and Technology（南方科技大学）；Shenzhen Research Institute of Trustworthy Autonomous Systems（深圳市可信自主系统研究院）；University of Macau（澳门大学）
Realizing human-like perception is a challenge in open driving scenarios due to corner cases and visual occlusions. To gather knowledge of rare and occluded instances, federated learning assisted connected autonomous vehicle (FLCAV) has been proposed, which leverages vehicular networks to establish federated deep neural networks (DNNs) from distributed data captured by vehicles and road sensors. Without the need of data aggregation, FLCAV preserves privacy while reducing communication costs compared with conventional centralized learning. However, it is challenging to determine the network res…
DP-EMAR: A Differentially Private Framework for Autonomous Model Weight Repair in Federated IoT Systems
Research Scholar, Department of CSE（计算机科学与工程系研究学者）；Professor, Department of CSE（计算机科学与工程系教授）；PES University, RR Nagar, Bangalore, India（印度班加罗尔RR Nagar的PES大学）
FedDSR: Federated Deep Supervision and Regularization Towards Autonomous Driving
Department of Electrical and Electronic Engineering, The University of Hong Kong（香港大学电子与电气工程系）；Department of Computer Science, City University of Hong Kong（城市大学计算机科学系）；Shenzhen International Center For Industrial And Applied Mathematics, Shenzhen Research Institute of Big Data（深圳国际工业与应用数学中心、深圳大数据研究院）
In this paper we envision a federated learning (FL) scenario in service of amending the performance of autonomous road vehicles, through a drone traffic monitor (DTM), that also acts as an orchestrator. Expecting non-IID data distribution, we focus on the issue of accelerating the learning of a particular class of critical object (CO), that may harm the nominal operation of an autonomous vehicle. This can be done through proper allocation of the wireless resources for addressing learner and data heterogeneity. Thus, we propose a reactive method for the allocation of wireless resources, that ha…
Universidad Loyola（洛约拉大学）；Liverpool John Moores University（利物浦约翰摩尔斯大学）；Technical School of Engineering of Seville（塞维利亚工程技术学院）
The preservation, monitoring, and control of water resources has been a major challenge in recent decades. Water resources must be constantly monitored to know the contamination levels of water. To meet this objective, this paper proposes a water monitoring system using autonomous surface vehicles, equipped with water quality sensors, based on a multimodal particle swarm optimization, and the federated learning technique, with Gaussian process as a surrogate model, the AquaFeL-PSO algorithm. The proposed monitoring system has two phases, the exploration phase and the exploitation phase. In the…
NEC Corporation（日本电气公司）；Samsung Electronics（三星电子）；Stanford University（斯坦福大学）；The University of Texas at Austin（德克萨斯大学奥斯汀分校）
Autonomous vehicles (AVs) must interact with a diverse set of human drivers in heterogeneous geographic areas. Ideally, fleets of AVs should share trajectory data to continually re-train and improve trajectory forecasting models from collective experience using cloud-based distributed learning. At the same time, these robots should ideally avoid uploading raw driver interaction data in order to protect proprietary policies (when sharing insights with other companies) or protect driver privacy from insurance companies. Federated learning (FL) is a popular mechanism to learn models in cloud serv…
FedEMA: Federated Exponential Moving Averaging with Negative Entropy Regularizer in Autonomous Driving
Department of Electrical and Electronic Engineering, The University of Hong Kong（香港大学电子与电气工程系）；Department of Computer Science and Engineering, Southern University of Science and Technology（南方科技大学计算机科学与工程系）；Shenzhen International Center For Industrial And Applied Mathematics, Shenzhen Research Institute of Big Data（深圳国际工业与应用数学中心、深圳大数据研究 institute）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究所）
