采用社会习俗的逆向引导自我游戏
Adversarially Guided Self-Play for Adopting Social Conventions
摘要：
机器人代理必须采用现有的社会惯例，以便成为有效的队友。这些社会惯例，例如在道路的右侧或左侧驾驶，是最优策略中的任意选择，但成功团队中的所有代理都必须使用相同的惯例。先前的工作已经确定了一种方法，该方法将自我游戏与从现有代理收集的成对输入-输出数据相结合，以便在不与它们交互的情况下学习它们的社会惯例。我们在这项工作的基础上，引入了一种称为对抗性自我游戏（ASP）的技术，该技术使用对抗性训练来塑造可能的学习策略空间，并大大提高了学习效率。ASP只需要添加不成对的数据：由社会惯例产生的输出数据集，没有相关的输入。理论分析揭示了ASP如何塑造政策空间和环境（当行为聚集或表现出一些其他结构），它提供了最大的好处。三个领域的实证结果证实了ASP的优势：它产生的模型，更紧密地匹配所需的社会习俗时，只有两个配对的数据点。
英文摘要：
Robotic agents must adopt existing social conventions in order to be effective teammates. These social conventions, such as driving on the right or left side of the road, are arbitrary choices among optimal policies, but all agents on a successful team must use the same convention. Prior work has identified a method of combining self-play with paired input-output data gathered from existing agents in order to learn their social convention without interacting with them. We build upon this work by introducing a technique called Adversarial Self-Play (ASP) that uses adversarial training to shape the space of possible learned policies and substantially improves learning efficiency. ASP only requires the addition of unpaired data: a dataset of outputs produced by the social convention without associated inputs. Theoretical analysis reveals how ASP shapes the policy space and the circumstances (when behaviors are clustered or exhibit some other structure) under which it offers the greatest benefits. Empirical results across three domains confirm ASP's advantages: it produces models that more closely match the desired social convention when given as few as two paired datapoints.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
