Simulated environments are a key piece in the success of Reinforcement Learning (RL), allowing practitioners and researchers to train decision making agents without running expensive experiments on real hardware. Simulators remain a security blind spot, however, enabling adversarial developers to alter the dynamics of their released simulators for malicious purposes. Therefore, in this work we highlight a novel threat, demonstrating how simulator dynamics can be exploited to stealthily implant action-level backdoors into RL agents. The backdoor then allows an adversary to reliably activate targeted actions in an agent upon observing a predefined ``trigger'', leading to potentially dangerous consequences. Traditional backdoor attacks are limited in their strong threat models, assuming the adversary has near full control over an agent's training pipeline, enabling them to both alter and observe agent's rewards. As these assumptions are infeasible to implement within a simulator, we propose a new attack ``Daze'' which is able to reliably and stealthily implant backdoors into RL agents trained for real world tasks without altering or even observing their rewards. We provide formal proof of Daze's effectiveness in guaranteeing attack success across general RL tasks along with extensive empirical evaluations on both discrete and continuous action space domains. We additionally provide the first example of RL backdoor attacks transferring to real, robotic hardware. These developments motivate further research into securing all components of the RL training pipeline to prevent malicious attacks.
DeepSweep: An Evaluation Framework for Mitigating DNN Backdoor Attacks using Data Augmentation
Tsinghua University（清华大学）；University of California San Diego（加利福尼亚大学圣迭戈分校）；Chongqing University（重庆大学）；Nanyang Technological University（南洋理工大学）；Texas A&M University Commerce（得克萨斯A&M大学康默斯分校）；The University of Texas at Dallas（得克萨斯大学达拉斯分校）
FilterFL: Knowledge Filtering-based Data-Free Backdoor Defense for Federated Learning
East China Normal University, Shanghai, China（华东师范大学）；Singapore Management University, Singapore（新加坡管理学院）；Nanyang Technological University, Singapore（南洋理工大学）；National University of Singapore, Singapore（国立新加坡大学）
BadAgent: Inserting and Activating Backdoor Attacks in LLM Agents
Zhengzhou University（郑州大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
Tsinghua University（清华大学）；Tsinghua Shenzhen International Graduate School（清华大学深圳国际研究生院）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Shenzhen Research Institute of Big Data（深圳大数据研究院）；Peng Cheng Laboratory（鹏城实验室）
Adaptive Attacks on Trusted Monitors Subvert AI Control Protocols
MATS；EPFL（瑞士联邦理工学院）；ELLIS Institute Tübingen & Max Planck Institute for Intelligent Systems（图宾根ELLIS研究所及图宾根马克斯·普朗克智能系统研究所）；Tübingen AI Center（图宾根人工智能中心）；University of Tübingen（图宾根大学）
