首页/论文/2001.03415

arXiv 2001.034152020-01cs.MAcs.LG

Multi-Agent Interactions Modeling with Correlated Policies

Shanghai Jiaotong University（上海交通大学）；Huawei Noah’s Ark Lab（华为诺亚方舟实验室）

Minghuan Liu, Ming Zhou, Weinan Zhang, Yuzheng Zhuang, Jun Wang, Wulong Liu, Yong Yu

备注：: 20 pages (10 pages of supplementary), 5 figures, Accepted by The Eighth International Conference on Learning Representations (ICLR 2020)

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

英文摘要相关论文

英文摘要：

In multi-agent systems, complex interacting behaviors arise due to the high correlations among agents. However, previous work on modeling multi-agent interactions from demonstrations is primarily constrained by assuming the independence among policies and their reward structures. In this paper, we cast the multi-agent interactions modeling problem into a multi-agent imitation learning framework with explicit modeling of correlated policies by approximating opponents' policies, which can recover agents' policies that can regenerate similar interactions. Consequently, we develop a Decentralized Adversarial Imitation Learning algorithm with Correlated policies (CoDAIL), which allows for decentralized training and execution. Various experiments demonstrate that CoDAIL can better regenerate complex interactions close to the demonstrators and outperforms state-of-the-art multi-agent imitation learning methods. Our code is available at \url{https://github.com/apexrl/CoDAIL}.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

AgentMixer：多智能体相关策略因子化

AgentMixer: Multi-Agent Correlated Policy Factorization

Aalto University（阿尔托大学）；University of Electronic Science and Technology of China（电子科技大学）

针对多智能体强化学习中CTDE方法独立决策难协调、模仿联合策略易出现非对称学习失败的问题，提出AgentMixer策略修正方法，通过非线性组合个体策略与个体-全局一致性机制实现分布式执行，收敛到ε-近似相关均衡，在多个基准上性能优于或持平SOTA。

2024-12-13cs.AIcs.MA

Metric Policy Representations for Opponent Modeling

Peking University（北京大学）

In multi-agent reinforcement learning, the inherent non-stationarity of the environment caused by other agents' actions posed significant difficulties for an agent to learn a good policy independently. One way to deal with non-stationarity is opponent modeling, by which the agent takes into consideration the influence of other agents' policies. Most existing work relies on predicting other agents' actions or goals, or discriminating between different policies. However, such modeling fails to capture the similarities and differences between policies simultaneously and thus cannot provide enough…

2022-06-21cs.AIcs.LGcs.MA

PolicySpace2: modeling markets and endogenous public policies

Policymakers decide on alternative policies facing restricted budgets and uncertain, ever-changing future. Designing public policies is further difficult due to the need to decide on priorities and handle effects across policies. Housing policies, specifically, involve heterogeneous characteristics of properties themselves and the intricacy of housing markets and the spatial context of cities. We propose PolicySpace2 (PS2) as an adapted and extended version of the open source PolicySpace agent-based model. PS2 is a computer simulation that relies on empirically detailed spatial data to model r…

2021-10-28cs.MAecon.GNphysics.soc-ph

TransAM：基于局部轨迹编码的多智能体系统Transformer智能体建模

TransAM: Transformer-Based Agent Modeling for Multi-Agent Systems via Local Trajectory Encoding

本文提出TransAM，一种基于Transformer的智能体建模方法，仅利用受控智能体的局部轨迹编码来捕获其他智能体策略，在合作、竞争和混合环境中验证了其生成强策略表示并提升情节回报的效果。

2025-08-04cs.AIcs.MA

Effect of intervention policies in an agent based spatial epidemic model of Gwalior

Covid-19 has ravaged the entire world and it may not be the last such to ravage the world. COMOKIT [3] is an agent based spatial modeling tool to study the effect of covid -19 in a geographical area by creating heterogenous synthetic agents and their behaviours. This paper presents comokit based case study on Gwalior region with respect to various intervention policies to curb the spread of the disease.

2022-01-19cs.MA

Modeling Internet-Scale Policies for Cleaning up Malware

An emerging consensus among policy makers is that interventions undertaken by Internet Service Providers are the best way to counter the rising incidence of malware. However, assessing the suitability of countermeasures at this scale is hard. In this paper, we use an agent-based model, called ASIM, to investigate the impact of policy interventions at the Autonomous System level of the Internet. For instance, we find that coordinated intervention by the 0.2%-biggest ASes is more effective than uncoordinated efforts adopted by 30% of all ASes. Furthermore, countermeasures that block malicious tr…

2012-02-17cs.CRcs.MAcs.NI

更多相关论文收起更多论文

Model-Based Opponent Modeling

Peking University（北京大学）

When one agent interacts with a multi-agent environment, it is challenging to deal with various opponents unseen before. Modeling the behaviors, goals, or beliefs of opponents could help the agent adjust its policy to adapt to different opponents. In addition, it is also important to consider opponents who are learning simultaneously or capable of reasoning. However, existing work usually tackles only one of the aforementioned types of opponents. In this paper, we propose model-based opponent modeling (MBOM), which employs the environment model to adapt to all kinds of opponents. MBOM simulate…

2022-06-21cs.AIcs.LGcs.MA

一种基于智能体的模型仿真方法：演示人口老龄化与社会服务政策对养老基金的影响及其长期社会经济后果

An Agent-based Model Simulation Approach to Demonstrate Effects of Aging Population and Social Service Policies on Pensions Fund and Its Long-term Socio-economic Consequences

本文提出利用基于智能体的建模（ABM）方法模拟人口老龄化与社会服务政策对养老基金的影响，展示了ABM在生成场景和评估适应性政策以支持社会政策制定方面的优势。

2025-04-01cs.MA

Analytically Modeling Unmanaged Intersections with Microscopic Vehicle Interactions

With the emergence of autonomous vehicles, it is important to understand their impact on the transportation system. However, conventional traffic simulations are time-consuming. In this paper, we introduce an analytical traffic model for unmanaged intersections accounting for microscopic vehicle interactions. The macroscopic property, i.e., delay at the intersection, is modeled as an event-driven stochastic dynamic process, whose dynamics encode the microscopic vehicle behaviors. The distribution of macroscopic properties can be obtained through either direct analysis or event-driven simulatio…

2018-09-06cs.MA

Modeling Strong and Human-Like Gameplay with KL-Regularized Search

Meta AI Research（Meta人工智能研究院）；CSAIL, MIT（麻省理工学院计算机科学与人工智能实验室）；Carnegie Mellon University（卡内基梅隆大学）

We consider the task of building strong but human-like policies in multi-agent decision-making problems, given examples of human behavior. Imitation learning is effective at predicting human actions but may not match the strength of expert humans, while self-play learning and search techniques (e.g. AlphaZero) lead to strong performance but may produce policies that are difficult for humans to understand and coordinate with. We show in chess and Go that regularizing search based on the KL divergence from an imitation-learned policy results in higher human prediction accuracy and stronger perfo…

2022-02-17cs.AIcs.GTcs.LG

利用ABM分析发展中国家的交通政策

Analyzing Transport Policies in Developing Countries with ABM

本文通过ABM仿真发展中国家通勤者的出行方式选择，评估公共交通免费政策的影响，发现该政策能促进公共交通使用并改善污染、事故率和行驶速度。

2024-04-30cs.MA

Learning Density-Based Correlated Equilibria for Markov Games

University of Electronic Science and Technology of China（电子科技大学）；The University of Auckland（奥克兰大学）

Correlated Equilibrium (CE) is a well-established solution concept that captures coordination among agents and enjoys good algorithmic properties. In real-world multi-agent systems, in addition to being in an equilibrium, agents' policies are often expected to meet requirements with respect to safety, and fairness. Such additional requirements can often be expressed in terms of the state density which measures the state-visitation frequencies during the course of a game. However, existing CE notions or CE-finding approaches cannot explicitly specify a CE with particular properties concerning s…

2023-02-16cs.AIcs.MA