arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2310.15414cs.AIcs.LGcs.MA
收藏

人机协作中的多样化惯例

Diverse Conventions for Human-AI Collaboration

Stanford University(斯坦福大学)

Bidipta Sarkar, Andy Shih, Dorsa Sadigh

AI总结:

本文提出一种通过最大化自我对局奖励并最小化交叉对局奖励来生成多样化惯例的技术,并引入混合对局机制确保策略善意,在Overcooked等协作博弈中适应人类惯例并超越人类水平表现。

摘要:

惯例在合作型多智能体博弈中对优异表现至关重要,因为它允许玩家在无需显式通信的情况下协调于共享策略。然而,标准的多智能体强化学习技术(如自我对局)会收敛到任意且缺乏多样性的惯例,导致与新伙伴交互时泛化能力差。在本研究中,我们提出一种生成多样化惯例的技术,通过 (1) 在自我对局中最大化其奖励,同时 (2) 在与已发现惯例进行交叉对局时最小化其奖励,从而促使惯例在语义上产生差异。为确保学习策略在交叉对局的对抗性优化下仍能善意行动,我们引入了混合对局机制:通过采样自我对局和交叉对局转移随机生成初始状态,玩家学习从该初始状态最大化自我对局奖励。我们在包括 Overcooked 在内的多种多智能体协作博弈上分析了该技术的优势,发现其能适应人类惯例,与真实用户组队时超越了人类水平表现。

英文摘要:

Conventions are crucial for strong performance in cooperative multi-agent games, because they allow players to coordinate on a shared strategy without explicit communication. Unfortunately, standard multi-agent reinforcement learning techniques, such as self-play, converge to conventions that are arbitrary and non-diverse, leading to poor generalization when interacting with new partners. In this work, we present a technique for generating diverse conventions by (1) maximizing their rewards during self-play, while (2) minimizing their rewards when playing with previously discovered conventions (cross-play), stimulating conventions to be semantically different. To ensure that learned policies act in good faith despite the adversarial optimization of cross-play, we introduce \emph{mixed-play}, where an initial state is randomly generated by sampling self-play and cross-play transitions and the player learns to maximize the self-play reward from this initial state. We analyze the benefits of our technique on various multi-agent collaborative games, including Overcooked, and find that our technique can adapt to the conventions of humans, surpassing human-level performance when paired with real users.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑