arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2309.01784cs.LGq-fin.TRstat.AP
收藏

INTAGS:交互式智能体引导的仿真

INTAGS: Interactive Agent-Guided Simulation

J.P. Morgan AI Research(摩根大通人工智能研究院);Georgia Institute of Technology(佐治亚理工学院)

Song Wei, Andrea Coletta, Svitlana Vyetrenko, Tucker Balch

AI总结:

针对多智能体系统仿真保真度不足的问题,提出INTAGS框架,通过基于因果推断的实时交互距离指标和强化学习策略梯度优化构建高保真模拟器,在股票市场仿真中生成数据优于条件WGAN方法。

摘要:

在许多涉及多智能体系统(MAS)的应用中,在将实验性(Exp)自主智能体部署到生产环境之前,必须在高保真模拟器中进行测试,以避免在现实世界中造成意外损失。此类模拟器充当环境背景(BG)智能体,被称为基于智能体的模拟器(ABS),旨在复现复杂的真实MAS。然而,由于此类系统具有序列和动态特性,开发逼真的ABS仍然具有挑战性。为了填补这一空白,我们提出了一种用于区分真实与合成多智能体系统的度量指标,该指标通过Exp智能体与BG智能体之间的实时交互来评估,以明确考虑系统的序列特性。具体而言,我们通过研究一系列BG智能体对环境状态演化的响应效果来表征系统/环境,并将这些效果的差异作为MAS距离度量指标;由于环境演化与先前的环境状态存在混杂关系,效果估计被转化为一个因果推断问题。重要的是,我们提出了交互式智能体引导的仿真(INTAGS)框架,通过优化这一新指标来构建逼真的ABS。为了适应任何具有交互式序列决策智能体的环境,INTAGS将模拟器表述为强化学习中的随机策略。此外,INTAGS利用策略梯度更新来绕过对所提指标的微分,从而支持多智能体环境中不可微的操作。通过大量实验,我们在股票市场仿真实例上证明了INTAGS的有效性。结果表明,与最先进的条件Wasserstein生成对抗网络方法相比,使用INTAGS校准模拟器可以生成更真实的市场数据。

英文摘要:

In many applications involving multi-agent system (MAS), it is imperative to test an experimental (Exp) autonomous agent in a high-fidelity simulator prior to its deployment to production, to avoid unexpected losses in the real-world. Such a simulator acts as the environmental background (BG) agent(s), called agent-based simulator (ABS), aiming to replicate the complex real MAS. However, developing realistic ABS remains challenging, mainly due to the sequential and dynamic nature of such systems. To fill this gap, we propose a metric to distinguish between real and synthetic multi-agent systems, which is evaluated through the live interaction between the Exp and BG agents to explicitly account for the systems' sequential nature. Specifically, we characterize the system/environment by studying the effect of a sequence of BG agents' responses to the environment state evolution and take such effects' differences as MAS distance metric; The effect estimation is cast as a causal inference problem since the environment evolution is confounded with the previous environment state. Importantly, we propose the Interactive Agent-Guided Simulation (INTAGS) framework to build a realistic ABS by optimizing over this novel metric. To adapt to any environment with interactive sequential decision making agents, INTAGS formulates the simulator as a stochastic policy in reinforcement learning. Moreover, INTAGS utilizes the policy gradient update to bypass differentiating the proposed metric such that it can support non-differentiable operations of multi-agent environments. Through extensive experiments, we demonstrate the effectiveness of INTAGS on an equity stock market simulation example. We show that using INTAGS to calibrate the simulator can generate more realistic market data compared to the state-of-the-art conditional Wasserstein Generative Adversarial Network approach.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑