建模半人马:序贯决策中的人机协同
Modeling the Centaur: Human-Machine Synergy in Sequential Decision Making
AI总结:
本文研究国际象棋序贯决策中人机团队的协同潜力,通过专家混合架构组合人类行为克隆模型Maia与自对弈强化学习引擎Leela,发现相对优势可被自动识别且专家贡献有限。
摘要:
集体智能领域研究团队如何取得比任何单个成员单独表现更好的结果。人机团队这一特殊情形在这方面带来了独特的挑战。例如,人类团队通常通过沟通来发现各自的相对优势从而实现协同,但如果团队伙伴是一个无法解释的深度神经网络,这种沟通方式便不可行。2005年至2008年间举办了一系列“自由式”国际象棋锦标赛,其中被称为“半人马”的人机团队表现优于单独的最强人类和最强机器。半人马棋手报告称,他们能够识别出自己与国际象棋程序之间的相对优势,即使该程序已经超越人类水平。受此启发,并借助近期开源模型,我们研究了国际象棋中类似人机的团队。我们将一个人类行为克隆模型(“Maia”)和一个纯自对弈强化学习训练的国际象棋引擎(“Leela”)通过专家混合(MoE)架构组合成一个团队。通过将研究问题聚焦于MoE的选择机制,我们得以在无需知识共享的情况下单独考察相对优势的提取问题。我们表明,原则上,在国际象棋这类复杂的序贯决策环境中,人类与机器之间具有很高的协同潜力。此外,我们表明,专家只能识别出这些相对优势中的一小部分,而且其领域专业知识在这一识别过程中的贡献很快就会饱和。这很可能是由于“知识的诅咒”现象所致。我们还训练了一个网络,在不具备国际象棋专业知识的情况下,利用强化学习来识别相对优势,其表现超过了专家。我们的实验在非对称团队中重复进行,在非对称团队中识别相对优势更具挑战性。我们的研究结果有助于集体智能和以人为中心的人工智能研究。
英文摘要:
The field of collective intelligence studies how teams can achieve better results than any of the team members alone. The special case of human-machine teams carries unique challenges in this regard. For example, human teams often achieve synergy by communicating to discover their relative advantages, which is not an option if the team partner is an unexplainable deep neural network. Between 2005-2008 a set of "freestyle" chess tournaments were held, in which human-machine teams known as "centaurs", outperformed the best humans and best machines alone. Centaur players reported that they identified relative advantages between themselves and their chess program, even though the program was superhuman. Inspired by this and leveraging recent open-source models, we study human-machine like teams in chess. A human behavioral clone ("Maia") and a pure self-play RL-trained chess engine ("Leela") were composed into a team using a Mixture of Experts (MoE) architecture. By directing our research question at the selection mechanism of the MoE, we could isolate the issue of extracting relative advantages without knowledge sharing. We show that in principle, there is high potential for synergy between human and machine in a complex sequential decision environment such as chess. Furthermore, we show that an expert can identify only a small part of these relative advantages, and that the contribution of its subject matter expertise in doing so saturates quickly. This is probably due to the "curse of knowledge" phenomenon. We also train a network to recognize relative advantages using reinforcement learning, without chess expertise, and it outdoes the expert. Our experiments are repeated in asymmetric teams, in which identifying relative advantages is more challenging. Our findings contribute to the study of collective intelligence and human-centric AI.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。