arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2108.03803cs.LGcs.MA
收藏

误导或误导:在多Agent通信强化学习中实现鲁棒性

Mis-spoke or mis-lead: Achieving Robustness in Multi-Agent Communicative Reinforcement Learning

Nanyang Technological University(南洋理工大学)

Wanqi Xue, Wei Qiu, Bo An, Zinovi Rabinovich, Svetlana Obraztsova, Chai Kiat Yeo

摘要:

多智能体通信强化学习(MACRL)的最新研究表明,多智能体协调可以通过允许智能体之间的通信来大大提高。与此同时,对抗性机器学习(ML)表明ML模型容易受到攻击。尽管ML算法的鲁棒性越来越受到关注,但如何在多智能体强化学习中实现鲁棒通信在很大程度上被忽视了。在本文中,我们系统地探讨了MACRL中的对抗性通信问题。我们的主要贡献有三个方面。首先,我们提出了一种有效的方法来执行攻击MACRL,通过学习一个模型来生成最佳的恶意消息。其次,我们提出了一种基于消息重构的防御方法,在消息攻击下保持多代理的协调。第三,我们将对抗性通信问题描述为两人零和博弈,并提出了一种博弈论方法R-MACRL来提高最坏情况下的防御性能。实验结果表明,许多国家的最先进的MACRL方法是容易受到消息攻击,和我们的方法可以显着提高其鲁棒性。

英文摘要:

Recent studies in multi-agent communicative reinforcement learning (MACRL) have demonstrated that multi-agent coordination can be greatly improved by allowing communication between agents. Meanwhile, adversarial machine learning (ML) has shown that ML models are vulnerable to attacks. Despite the increasing concern about the robustness of ML algorithms, how to achieve robust communication in multi-agent reinforcement learning has been largely neglected. In this paper, we systematically explore the problem of adversarial communication in MACRL. Our main contributions are threefold. First, we propose an effective method to perform attacks in MACRL, by learning a model to generate optimal malicious messages. Second, we develop a defence method based on message reconstruction, to maintain multi-agent coordination under message attacks. Third, we formulate the adversarial communication problem as a two-player zero-sum game and propose a game-theoretical method R-MACRL to improve the worst-case defending performance. Empirical results demonstrate that many state-of-the-art MACRL methods are vulnerable to message attacks, and our method can significantly improve their robustness.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑