通过观察增强的自我游戏学习现有的社会习俗
Learning Existing Social Conventions via Observationally Augmented Self-Play
摘要：
为了让人工智能体有效地与人协调，它们必须与现有的约定保持一致（例如，如何在交通中导航，说哪种语言，或如何与队友协调）。一个群体的约定可以被看作是协调博弈中的一种均衡选择。我们考虑的问题，代理学习的政策，协调游戏在一个模拟的环境中，然后使用这个政策时，它进入一个现有的组。当有多个可能的约定时，我们表明通过多智能体强化学习（MARL）学习策略可能会找到在训练时实现高回报但无法与代理进入的真实组协调的策略。我们假设访问少量的样本的行为，从真正的公约，并表明，我们可以增加MARL的目标，以帮助它找到符合真正的集团的公约的政策。在文献中的三种环境中--交通、沟通和团队协调--我们观察到，用少量的模仿学习来增强MARL，大大增加了MARL发现的策略与现有社会惯例很好地吻合的概率。我们表明，即使在一个环境中，标准的训练方法很少找到真正的公约代理的合作伙伴，这也是可行的。
英文摘要：
In order for artificial agents to coordinate effectively with people, they must act consistently with existing conventions (e.g. how to navigate in traffic, which language to speak, or how to coordinate with teammates). A group's conventions can be viewed as a choice of equilibrium in a coordination game. We consider the problem of an agent learning a policy for a coordination game in a simulated environment and then using this policy when it enters an existing group. When there are multiple possible conventions we show that learning a policy via multi-agent reinforcement learning (MARL) is likely to find policies which achieve high payoffs at training time but fail to coordinate with the real group into which the agent enters. We assume access to a small number of samples of behavior from the true convention and show that we can augment the MARL objective to help it find policies consistent with the real group's convention. In three environments from the literature - traffic, communication, and team coordination - we observe that augmenting MARL with a small amount of imitation learning greatly increases the probability that the strategy found by MARL fits well with the existing social convention. We show that this works even in an environment where standard training methods very rarely find the true convention of the agent's partners.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
