arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 2210.06012cs.AIcs.MA
收藏

Phantom——一个用于建模复杂系统的强化学习驱动的多智能体框架

Phantom -- A RL-driven multi-agent framework to model complex systems

J.P. Morgan AI Research(摩根大通人工智能研究院);Sutter Hill Ventures(萨特希尔风险投资公司)

Leo Ardon, Jared Vann, Deepeka Garg, Tom Spooner, Sumitra Ganesh

AI总结:

本文提出了开源框架Phantom,通过提供兼容多智能体强化学习(MARL)的接口与特征,弥合了基于智能体建模(ABM)与MARL之间的差距,简化了复杂多智能体系统的建模规范。

摘要:

基于智能体的建模(ABM)是一种计算方法,通过指定系统中自主决策组件或智能体的行为,并让系统动态从它们的交互中涌现出来,从而对复杂系统进行建模。多智能体强化学习(MARL)领域的最新进展使得研究多个智能体同时学习的复杂环境的均衡成为可能。然而,大多数ABM框架并非原生支持RL,即它们没有提供与使用MARL学习智能体行为兼容的概念和接口。在本文中,我们引入了一个新的开源框架Phantom,以弥合ABM和MARL之间的差距。Phantom是一个RL驱动的框架,用于对复杂多智能体系统(包括但不限于经济系统和市场)进行基于智能体的建模。该框架旨在提供工具,以兼容MARL的方式简化ABM规范——包括对动态部分可观测性、智能体效用函数、智能体偏好或类型的异质性以及智能体行动顺序的约束(例如Stackelberg博弈或更复杂的轮流行动环境)进行编码的特征。在本文中,我们介绍了这些特征、它们的设计原理,并展示了利用该框架的两个新环境。

英文摘要:

Agent based modelling (ABM) is a computational approach to modelling complex systems by specifying the behaviour of autonomous decision-making components or agents in the system and allowing the system dynamics to emerge from their interactions. Recent advances in the field of Multi-agent reinforcement learning (MARL) have made it feasible to study the equilibrium of complex environments where multiple agents learn simultaneously. However, most ABM frameworks are not RL-native, in that they do not offer concepts and interfaces that are compatible with the use of MARL to learn agent behaviours. In this paper, we introduce a new open-source framework, Phantom, to bridge the gap between ABM and MARL. Phantom is an RL-driven framework for agent-based modelling of complex multi-agent systems including, but not limited to economic systems and markets. The framework aims to provide the tools to simplify the ABM specification in a MARL-compatible way - including features to encode dynamic partial observability, agent utility functions, heterogeneity in agent preferences or types, and constraints on the order in which agents can act (e.g. Stackelberg games, or more complex turn-taking environments). In this paper, we present these features, their design rationale and present two new environments leveraging the framework.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑