arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.07775cs.AIcs.CR
收藏

针对协作多智能体深度强化学习的时空隐蔽后门攻击

A Spatiotemporal Stealthy Backdoor Attack against Cooperative Multi-Agent Deep Reinforcement Learning

School of Cybersecurity, Northwestern Polytechnical University(西北工业大学网络空间安全学院);Research & Development Institute of Northwestern Polytechnical University in Shenzhen(西北工业大学深圳研究院)

Yinbo Yu, Saihao Yan, Jiajia Liu

AI总结:

针对协作多智能体深度强化学习现有后门攻击隐蔽性不足等问题,提出仅植入单智能体的时空行为触发后门攻击,在SMAC环境的VDN、QMIX算法上实现高成功率与低性能影响。

摘要:

近期研究表明,协作多智能体深度强化学习(c-MADRL)面临后门攻击的威胁。一旦观测到后门触发器,智能体将执行异常动作,导致任务失败或达成恶意目标。但现有后门方案存在诸多问题,例如固定视觉触发模式隐蔽性不足、后门需借助额外网络训练或激活,或是所有智能体均被植入后门。为此,本文提出一种针对c-MADRL的新型后门攻击,仅在单个智能体中植入后门即可攻击整个多智能体团队。首先,我们引入对抗性时空行为模式作为后门触发器,而非人工注入的固定视觉模式或瞬时状态,并可控制攻击持续时长,该方法能保障植入后门的隐蔽性与实用性。其次,训练阶段通过奖励反转和单向引导篡改被植入后门智能体的原始奖励函数,确保其对整个团队产生负面影响。我们在主流c-MADRL环境SMAC中,针对VDN和QMIX这两种经典c-MADRL算法评估了所提后门攻击的效果。实验结果表明,该后门攻击可实现91.6%的高攻击成功率,同时保持3.7%的低干净性能波动率。

英文摘要:

Recent studies have shown that cooperative multi-agent deep reinforcement learning (c-MADRL) is under the threat of backdoor attacks. Once a backdoor trigger is observed, it will perform abnormal actions leading to failures or malicious goals. However, existing proposed backdoors suffer from several issues, e.g., fixed visual trigger patterns lack stealthiness, the backdoor is trained or activated by an additional network, or all agents are backdoored. To this end, in this paper, we propose a novel backdoor attack against c-MADRL, which attacks the entire multi-agent team by embedding the backdoor only in a single agent. Firstly, we introduce adversary spatiotemporal behavior patterns as the backdoor trigger rather than manual-injected fixed visual patterns or instant status and control the attack duration. This method can guarantee the stealthiness and practicality of injected backdoors. Secondly, we hack the original reward function of the backdoored agent via reward reverse and unilateral guidance during training to ensure its adverse influence on the entire team. We evaluate our backdoor attacks on two classic c-MADRL algorithms VDN and QMIX, in a popular c-MADRL environment SMAC. The experimental results demonstrate that our backdoor attacks are able to reach a high attack success rate (91.6\%) while maintaining a low clean performance variance rate (3.7\%).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑