基于选择性状态空间模型的多智能体强化学习
Multi-Agent Reinforcement Learning with Selective State-Space Models
AI总结：
本文提出融合标准、双向及新型交叉注意力Mamba块的Multi-Agent Mamba，在保持MAT性能的同时显著提升多智能体场景的可扩展性。
摘要：
Transformer模型已在众多领域取得成功，包括多智能体强化学习（MARL）领域，其中Multi-Agent Transformer（MAT）已成为该领域的领先算法。然而，Transformer模型的一个显著缺点是其相对于输入规模呈二次方的计算复杂度，这使其在扩展到更大输入时计算成本高昂。这一限制制约了MAT在包含大量智能体的环境中的可扩展性。近来，状态空间模型（SSM）因其计算效率而受到关注，但它们在MARL中的应用仍未得到探索。在本工作中，我们研究了近期的SSM——Mamba在MARL中的使用，并评估它是否能够在显著提升效率的同时达到与MAT相当的性能。我们提出了MAT的一个修改版本，该版本融合了标准Mamba块和双向Mamba块，以及一种新颖的“交叉注意力”Mamba块。大量测试表明，我们的Multi-Agent Mamba（MAM）在多个标准多智能体环境中达到了与MAT相当的性能，同时在更大规模智能体场景中提供了更优的可扩展性。这对MARL社区具有重要意义，因为它表明SSM可以在不损失性能的情况下取代Transformer，同时还支持更有效地扩展到更多智能体。我们的项目页面位于https://sites.google.com/view/multi-agent-mamba。
英文摘要：
The Transformer model has demonstrated success across a wide range of domains, including in Multi-Agent Reinforcement Learning (MARL) where the Multi-Agent Transformer (MAT) has emerged as a leading algorithm in the field. However, a significant drawback of Transformer models is their quadratic computational complexity relative to input size, making them computationally expensive when scaling to larger inputs. This limitation restricts MAT's scalability in environments with many agents. Recently, State-Space Models (SSMs) have gained attention due to their computational efficiency, but their application in MARL remains unexplored. In this work, we investigate the use of Mamba, a recent SSM, in MARL and assess whether it can match the performance of MAT while providing significant improvements in efficiency. We introduce a modified version of MAT that incorporates standard and bi-directional Mamba blocks, as well as a novel "cross-attention" Mamba block. Extensive testing shows that our Multi-Agent Mamba (MAM) matches the performance of MAT across multiple standard multi-agent environments, while offering superior scalability to larger agent scenarios. This is significant for the MARL community, because it indicates that SSMs could replace Transformers without compromising performance, whilst also supporting more effective scaling to higher numbers of agents. Our project page is available at https://sites.google.com/view/multi-agent-mamba .
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
