While real-world applications of reinforcement learning are becoming popular, the security and robustness of RL systems are worthy of more attention and exploration. In particular, recent works have revealed that, in a multi-agent RL environment, backdoor trigger actions can be injected into a victim agent (a.k.a. Trojan agent), which can result in a catastrophic failure as soon as it sees the backdoor trigger action. To ensure the security of RL agents against malicious backdoors, in this work, we propose the problem of Backdoor Detection in a multi-agent competitive reinforcement learning system, with the objective of detecting Trojan agents as well as the corresponding potential trigger actions, and further trying to mitigate their Trojan behavior. In order to solve this problem, we propose PolicyCleanse that is based on the property that the activated Trojan agents accumulated rewards degrade noticeably after several timesteps. Along with PolicyCleanse, we also design a machine unlearning-based approach that can effectively mitigate the detected backdoor. Extensive experiments demonstrate that the proposed methods can accurately detect Trojan agents, and outperform existing backdoor mitigation baseline approaches by at least 3% in winning rate across various types of agents and environments.
BehaviorGuard: Online Backdoor Defense for Deep Reinforcement Learning
College of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics（南京航空航天大学人工智能学院）；School of Cybersecurity, Northwestern Polytechnical University（西北工业大学网络安全学院）；School of Computer Science and Technology, Harbin Institute of Technology（哈尔滨工业大学计算机科学与技术学院）；Department of Electronic and Electrical Engineering, University College London（伦敦大学学院电子与电气工程系）
Mutual Information Guided Backdoor Mitigation for Pre-trained Encoders
State Key Laboratory for Novel Software Technology, Nanjing University（南京大学新型软件技术国家重点实验室）；College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算机与数据科学学院）
Fusing Pruned and Backdoored Models: Optimal Transport-based Data-free Backdoor Mitigation
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Shenzhen Research Institute of Big Data（深圳大数据研究院）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））
FL-PBM: Pre-Training Backdoor Mitigation for Federated Learning
Polytechnique Montréal（蒙特利尔理工学院）；Khalifa University（哈利法大学）；Concordia Institute for Information Systems Engineering, Concordia University（康考迪亚大学康考迪亚信息系统工程研究所）
Defending the Edge: Representative-Attention Defense against Backdoor Attacks in Federated Learning
Dept. of Computer Science Faculty of Engineering University of Manchester Manchester, United Kingdom；Dept. of Computer Science Faculty of Engineering University of Manchester \& MBZUAI Abu Dhabi, United Arab Emirates；Dept. of Computer Science Faculty of Engineering University of Manchester \& KU Leuven Manchester, United Kingdom
PromptFix: Few-shot Backdoor Removal via Adversarial Prompt Tuning
School of Information Science & Technology, Pennsylvania State University（宾夕法尼亚州立大学信息科学与技术学院）；Department of Computer Science, Stony Brook University（石溪大学计算机科学系）
Analyzing and Mitigating Poisoning Attacks on Flow-Based Intrusion Detection
CortAIx Labs, Thales Deutschland（Thales Deutschland）；Grey Solutions；University of Osnabrück（奥斯纳布吕克大学）；Karlsruhe Institute of Technology (KIT)（卡尔斯鲁厄理工学院）；Ulm University（乌尔姆大学）
