亲社会学习代理比自私学习代理更好地解决广义猎鹿问题
Prosocial learning agents solve generalized Stag Hunts better than selfish ones
摘要：
深度强化学习已经成为构建能够进入复杂多智能体情况并通过经验改进其策略的智能体的重要范式。一种常用的技术是反应式训练-应用标准RL方法，同时将其他代理视为学习者环境的一部分。众所周知，在一般和游戏中，反应式训练会导致代理群体收敛到低效的结果。我们专注于这样一类环境：猎鹿游戏。在这种情况下，代理人要么选择一个有风险的合作策略（如果双方都选择它，则会导致高收益，但单独尝试它的代理人的收益较低），要么选择一个安全的策略（无论如何都会导致安全的收益）。我们问，我们如何才能改变一个单一的代理的学习规则，以改善其结果在雄鹿狩猎，包括其他反应式学习者。我们扩展了多智能体强化学习中奖励塑造的现有工作，并表明使单个智能体亲社会，也就是说，使他们关心他们的合作伙伴的奖励可以增加群体收敛到好结果的概率。因此，即使我们控制了一个群体中的一个个体，让那个个体亲社会也能增加我们的个体的长期收益。我们的实验表明，这一结果结转到各种更复杂的环境与雄鹿狩猎样的动态，包括代理必须从原始输入像素学习。
英文摘要：
Deep reinforcement learning has become an important paradigm for constructing agents that can enter complex multi-agent situations and improve their policies through experience. One commonly used technique is reactive training - applying standard RL methods while treating other agents as a part of the learner's environment. It is known that in general-sum games reactive training can lead groups of agents to converge to inefficient outcomes. We focus on one such class of environments: Stag Hunt games. Here agents either choose a risky cooperative policy (which leads to high payoffs if both choose it but low payoffs to an agent who attempts it alone) or a safe one (which leads to a safe payoff no matter what). We ask how we can change the learning rule of a single agent to improve its outcomes in Stag Hunts that include other reactive learners. We extend existing work on reward-shaping in multi-agent reinforcement learning and show that that making a single agent prosocial, that is, making them care about the rewards of their partners can increase the probability that groups converge to good outcomes. Thus, even if we control a single agent in a group making that agent prosocial can increase our agent's long-run payoff. We show experimentally that this result carries over to a variety of more complex environments with Stag Hunt-like dynamics including ones where agents must learn from raw input pixels.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
