In cooperative multi-agent reinforcement learning (Co-MARL), a team of agents must jointly optimize the team's long-term rewards to learn a designated task. Optimizing rewards as a team often requires inter-agent communication and data sharing, leading to potential privacy implications. We assume privacy considerations prohibit the agents from sharing their environment interaction data. Accordingly, we propose Privacy-Engineered Value Decomposition Networks (PE-VDN), a Co-MARL algorithm that models multi-agent coordination while provably safeguarding the confidentiality of the agents' environment interaction data. We integrate three privacy-engineering techniques to redesign the data flows of the VDN algorithm, an existing Co-MARL algorithm that consolidates the agents' environment interaction data to train a central controller that models multi-agent coordination, and develop PE-VDN. In the first technique, we design a distributed computation scheme that eliminates Vanilla VDN's dependency on sharing environment interaction data. Then, we utilize a privacy-preserving multi-party computation protocol to guarantee that the data flows of the distributed computation scheme do not pose new privacy risks. Finally, we enforce differential privacy to preempt inference threats against the agents' training data, past environment interactions, when they take actions based on their neural network predictions. We implement PE-VDN in StarCraft Multi-Agent Competition (SMAC) and show that it achieves 80% of Vanilla VDN's win rate while maintaining differential privacy levels that provide meaningful privacy guarantees. The results demonstrate that PE-VDN can safeguard the confidentiality of agents' environment interaction data without sacrificing multi-agent coordination.
An Automated Reinforcement Learning Reward Design Framework with Large Language Model for Cooperative Platoon Coordination
Shanghai Research Institute for Intelligent Autonomous Systems, Tongji University(上海智能自主系统研究院,同济大学);Department of Control Science and Engineering, Tongji University(控制科学与工程系,同济大学)
Pareto Actor-Critic for Communication and Computation Co-Optimization in Non-Cooperative Federated Learning Services
College of Information Science and Electronic Engineering, Zhejiang University(浙江大学信息科学与电子工程学院);Shenzhen CyberAray Network Technology Co., Ltd, China(深圳 CyberAray 网络技术有限公司);Information and Communication Branch of State Grid Hebei Electric Power Co., Ltd, China(国网河北电力信息通信分部);Zhejiang Lab as well as Zhejiang University(浙江实验室以及浙江大学)
Privacy-Aware Agent Collaboration for Dynamic VR Slice Management in 6G SD-RAN
CCAS Department, School of Engineering, NewGiza University (NGU)(计算机与软件工程学院,新吉扎大学);Electronics and Communications, Helwan Univ. Cairo, Egypt(电子与通信,开罗海威大学,埃及)