在马尔可夫博弈中伪造纳什均衡的数据投毒
Data Poisoning to Fake a Nash Equilibrium in Markov Games
AI总结：
该研究刻画了离线多智能体强化学习的数据投毒攻击，提出唯一纳什集概念，通过线性规划高效计算最优投毒策略，为设计鲁棒MARL算法奠定基础。
摘要：
我们刻画了针对多智能体强化学习（MARL）的离线数据投毒攻击：攻击者可篡改数据集，试图为两人零和马尔可夫博弈植入一个（可能是虚构的）唯一马尔可夫完美纳什均衡。我们提出了唯一纳什集的概念，即由其Q函数定义、且某一特定联合策略为唯一纳什均衡的博弈集合。唯一纳什集是投毒攻击的核心，因为攻击成功当且仅当数据投毒将所有可信博弈都推入该集合内。唯一纳什集将逆强化学习中常用的奖励多面体推广到了MARL场景。对于零和马尔可夫博弈，逆纳什集与由数据诱导的可信博弈集合均为Q函数空间中的多面体。我们提出了一种线性规划方法，可高效计算最优投毒攻击。我们的工作揭示了离线MARL数据投毒攻击的结构，这是设计更鲁棒的MARL算法前的必要步骤。
英文摘要：
We characterize offline data poisoning attacks on Multi-Agent Reinforcement Learning (MARL), where an attacker may change a data set in an attempt to install a (potentially fictitious) unique Markov-perfect Nash equilibrium for a two-player zero-sum Markov game. We propose the unique Nash set, namely the set of games, specified by their Q functions, with a specific joint policy being the unique Nash equilibrium. The unique Nash set is central to poisoning attacks because the attack is successful if and only if data poisoning pushes all plausible games inside the set. The unique Nash set generalizes the reward polytope commonly used in inverse reinforcement learning to MARL. For zero-sum Markov games, both the inverse Nash set and the set of plausible games induced by data are polytopes in the Q function space. We exhibit a linear program to efficiently compute the optimal poisoning attack. Our work sheds light on the structure of data poisoning attacks on offline MARL, a necessary step before one can design more robust MARL algorithms.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
