arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.03575cs.AIcs.HC
收藏

面向大规模多人游戏中的人-AI对齐

Toward Human-AI Alignment in Large-Scale Multi-Player Games

Microsoft Research(微软研究院);Massachusetts Institute of Technology(麻省理工学院);New York University(纽约大学);Mila, University of Montreal(蒙特利尔大学Mila研究所)

Sugandha Sharma, Guy Davidson, Khimya Khetarpal, Anssi Kanervisto, Udit Arora, Katja Hofmann, Ida Momennejad

AI总结:

本研究针对复杂多人游戏中的人-AI对齐问题,提出基于可解释任务集框架的评估方法,通过分析超10万场Bleeding Edge人类对局数据构建行为流形,对比生成式预训练因果Transformer训练的AI与人类的行为差异,为多人游戏的人-AI对齐提供可量化的可解释评估框架。

摘要:

在复杂多智能体游戏中实现人-AI对齐,对于打造可提升游戏体验的可信AI智能体至关重要。我们提出一种利用可解释任务集框架评估该对齐性的方法,聚焦于高层行为任务而非底层策略。该方法包含三个组成部分。第一,我们分析了来自Xbox旗下Bleeding Edge的海量人类游戏数据(超10万场对局),揭示了复杂任务空间中的行为模式。该任务空间作为行为流形的基集,承载了多个可解释维度:战斗-逃跑、探索-利用以及单人-多智能体。第二,我们使用生成式预训练因果Transformer训练了一个游玩Bleeding Edge的AI智能体,并对其行为进行测量。第三,我们将人类与AI的游戏行为投影到所提出的行为流形上进行对比分析。这使得我们能够将策略差异解读为更高层级的行为概念,例如我们发现人类玩家在战斗-逃跑和探索-利用行为上表现出差异性,而AI玩家则趋于一致。此外,AI智能体主要进行单人游玩,而人类经常参与合作与竞争类的多智能体模式。这些显著差异凸显了在人对齐的应用场景中,对AI进行可解释评估、设计与整合的必要性。我们的研究推动了AI尤其是生成式AI研究领域的对齐讨论,为多人游戏场景下可解释的人-智能体对齐提供了可量化的框架。

英文摘要:

Achieving human-AI alignment in complex multi-agent games is crucial for creating trustworthy AI agents that enhance gameplay. We propose a method to evaluate this alignment using an interpretable task-sets framework, focusing on high-level behavioral tasks instead of low-level policies. Our approach has three components. First, we analyze extensive human gameplay data from Xbox's Bleeding Edge (100K+ games), uncovering behavioral patterns in a complex task space. This task space serves as a basis set for a behavior manifold capturing interpretable axes: fight-flight, explore-exploit, and solo-multi-agent. Second, we train an AI agent to play Bleeding Edge using a Generative Pretrained Causal Transformer and measure its behavior. Third, we project human and AI gameplay to the proposed behavior manifold to compare and contrast. This allows us to interpret differences in policy as higher-level behavioral concepts, e.g., we find that while human players exhibit variability in fight-flight and explore-exploit behavior, AI players tend towards uniformity. Furthermore, AI agents predominantly engage in solo play, while humans often engage in cooperative and competitive multi-agent patterns. These stark differences underscore the need for interpretable evaluation, design, and integration of AI in human-aligned applications. Our study advances the alignment discussion in AI and especially generative AI research, offering a measurable framework for interpretable human-agent alignment in multiplayer gaming.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑