CivRealm：决策智能体在《文明》中的学习与推理奥德赛
CivRealm: A Learning and Reasoning Odyssey in Civilization for Decision-Making Agents
AI总结：
针对现有交互环境重学习轻推理的问题，本文提出受《文明》启发的决策智能体环境CivRealm，其具备不完全信息、玩家数动态变化等复杂特性，为两类智能体提供接口，初步实验显示现有RL和LLM智能体在完整游戏中进展有限，为智能体学习与推理研究提供新挑战。
摘要：
决策智能体的泛化能力包含两个核心要素：从过往经验中学习，以及在全新情境中进行推理。然而，当前大多数交互环境主要侧重于学习，往往以牺牲推理的复杂性为代价。本文提出了受《文明》游戏启发的环境CivRealm。《文明》与人类历史和社会高度契合，因此需要复杂的学习能力，而其不断变化的局势则要求强大的推理能力才能实现泛化。具体而言，CivRealm构建了一个玩家数量动态变化的不完全信息非零和博弈；它具备大量复杂特性，对智能体应对需要外交与谈判技巧的开放式随机环境构成了挑战。在CivRealm中，我们为两类典型智能体提供了接口：侧重学习的基于张量的智能体，以及侧重推理的基于语言的智能体。为推动后续研究，我们给出了两种范式的初步结果：经典的基于RL（强化学习）的智能体在迷你游戏中表现尚可，但基于RL和基于LLM（大语言模型）的智能体在完整游戏中都难以取得实质性进展。总体而言，CivRealm为决策智能体提出了一项独特的学习与推理挑战。代码可在https://github.com/bigai-ai/civrealm获取。
英文摘要：
The generalization of decision-making agents encompasses two fundamental elements: learning from past experiences and reasoning in novel contexts. However, the predominant emphasis in most interactive environments is on learning, often at the expense of complexity in reasoning. In this paper, we introduce CivRealm, an environment inspired by the Civilization game. Civilization's profound alignment with human history and society necessitates sophisticated learning, while its ever-changing situations demand strong reasoning to generalize. Particularly, CivRealm sets up an imperfect-information general-sum game with a changing number of players; it presents a plethora of complex features, challenging the agent to deal with open-ended stochastic environments that require diplomacy and negotiation skills. Within CivRealm, we provide interfaces for two typical agent types: tensor-based agents that focus on learning, and language-based agents that emphasize reasoning. To catalyze further research, we present initial results for both paradigms. The canonical RL-based agents exhibit reasonable performance in mini-games, whereas both RL- and LLM-based agents struggle to make substantial progress in the full game. Overall, CivRealm stands as a unique learning and reasoning challenge for decision-making agents. The code is available at https://github.com/bigai-ai/civrealm.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
