学习与社会理性智能体群体合作的复杂性
On the Complexity of Learning to Cooperate with Populations of Socially Rational Agents
AI总结:
本文研究在有限重复一般和矩阵博弈中学习与个体理性且能达到 Pareto 有效均衡效用的智能体群体合作的样本复杂度,证明仅靠群体假设无法实现零样本合作,并给出远优于朴素模仿学习归约的样本上下界。
摘要:
部署在现实世界中的人工智能智能体需要具备与人类(以及其他异构 AI 智能体)可靠合作的能力。为了对成功合作提供形式化保证,我们必须对伙伴智能体可能如何行为做出一些假设。任何现实的假设集合都必须考虑到其他智能体可能与我们自己的智能体一样具有适应性这一事实。在这项工作中,我们考虑在有限重复的两人一般和矩阵博弈中与一个智能体群体合作的问题,其中效用为私有信息。在此类设定下有两个自然的假设:1)群体中的所有智能体都是个体理性学习者;2)当群体中任意两个成员被配对在一起时,它们以高概率至少获得与在某些 Pareto 有效均衡策略下相同的效用。我们的结果首先表明,仅凭这些假设不足以确保与目标群体成员实现零样本合作。因此,我们考虑利用群体成员之间相互交互的先验观测来学习与该群体合作策略的问题。我们给出了学习有效合作策略所需样本数量的上界和下界。最重要的是,我们证明了这些界可以远强于将问题“朴素”归约为模仿学习问题所产生的界。
英文摘要:
Artificially intelligent agents deployed in the real-world will require the ability to reliably \textit{cooperate} with humans (as well as other, heterogeneous AI agents). To provide formal guarantees of successful cooperation, we must make some assumptions about how partner agents could plausibly behave. Any realistic set of assumptions must account for the fact that other agents may be just as adaptable as our agent is. In this work, we consider the problem of cooperating with a \textit{population} of agents in a finitely-repeated, two player general-sum matrix game with private utilities. Two natural assumptions in such settings are that: 1) all agents in the population are individually rational learners, and 2) when any two members of the population are paired together, with high-probability they will achieve at least the same utility as they would under some Pareto efficient equilibrium strategy. Our results first show that these assumptions alone are insufficient to ensure \textit{zero-shot} cooperation with members of the target population. We therefore consider the problem of \textit{learning} a strategy for cooperating with such a population using prior observations its members interacting with one another. We provide upper and lower bounds on the number of samples needed to learn an effective cooperation strategy. Most importantly, we show that these bounds can be much stronger than those arising from a "naive'' reduction of the problem to one of imitation learning.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。