arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2202.03558cs.LGcs.AI
收藏

更有效地攻击 c-MARL:一种数据驱动的方法

Attacking c-MARL More Effectively: A Data Driven Approach

IBM Research(IBM研究院);MIT-IBM Watson AI Lab(MIT-IBM沃森人工智能实验室);University of California San Diego(加州大学圣地亚哥分校)

Nhan H. Pham, Lam M. Nguyen, Jie Chen, Hoang Thanh Lam, Subhro Das, Tsui-Wei Weng

AI总结:

本文提出基于模型的 c-MBA 方法评估合作多智能体强化学习(c-MARL)的鲁棒性,通过构造对抗状态扰动并引入首个受害智能体选择策略和数据驱动的目标故障状态定义,在无需专家知识的情况下实现了持续优于基线的对抗攻击。

摘要:

近年来,针对合作多智能体强化学习(c-MARL)开发了大量的方法。然而,c-MARL 智能体针对对抗攻击的鲁棒性却鲜有探索。在本文中,我们提出通过一种基于模型的方法(名为 c-MBA)来评估 c-MARL 智能体的鲁棒性。我们提出的公式能够为 c-MARL 智能体构造比现有无模型方法更强得多的对抗状态扰动,从而降低团队总奖励。此外,我们提出了首个受害智能体选择策略以及首个数据驱动方法来定义目标故障状态,这两者均使我们能够在不具备底层环境专家知识的情况下,开发出更强的对抗攻击。我们在两个代表性 MARL 基准上的数值实验展示了我们的方法相对于其他基线的优势:我们的基于模型的攻击在所有测试环境中均持续优于其他基线。

英文摘要:

In recent years, a proliferation of methods were developed for cooperative multi-agent reinforcement learning (c-MARL). However, the robustness of c-MARL agents against adversarial attacks has been rarely explored. In this paper, we propose to evaluate the robustness of c-MARL agents via a model-based approach, named c-MBA. Our proposed formulation can craft much stronger adversarial state perturbations of c-MARL agents to lower total team rewards than existing model-free approaches. In addition, we propose the first victim-agent selection strategy and the first data-driven approach to define targeted failure states where each of them allows us to develop even stronger adversarial attack without the expert knowledge to the underlying environment. Our numerical experiments on two representative MARL benchmarks illustrate the advantage of our approach over other baselines: our model-based attack consistently outperforms other baselines in all tested environments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑