arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2410.01706cs.LGcs.AIcs.MA
收藏

Sable:一种用于多智能体强化学习的高性能、高效且可扩展的序列模型

Sable: a Performant, Efficient and Scalable Sequence Model for MARL

InstaDeep

Omayma Mahjoub, Sasha Abramowitz, Ruan de Kock, Wiem Khlifi, Simon du Toit, Jemma Daniel, Louay Ben Nessir, Louise Beyers, Claude Formanek, Liam Clark, Arnu Pretorius

AI总结:

Sable是一种基于保留机制的高性能、内存高效且可扩展的MARL序列模型,在45个任务中34个超越最先进方法,支持千级智能体且内存线性增长。

摘要:

随着多智能体强化学习(MARL)朝着解决更大、更复杂问题的方向发展,算法展现出(1)强性能、(2)内存效率和(3)可扩展性这三个关键属性变得越来越重要。在这项工作中,我们引入了Sable,一种高性能、内存高效且可扩展的MARL序列建模方法。Sable通过适配Retentive Networks(Sun等人,2023)中的保留机制,实现对多智能体观测的计算高效处理,并具备用于时间推理的长上下文记忆。通过在六个不同环境中的广泛评估,我们展示了Sable如何在大量不同任务(45个测试任务中的34个)中显著优于现有最先进方法。此外,随着智能体数量的增加,Sable保持性能,能够处理超过一千个智能体的环境,同时内存使用呈线性增长。最后,我们进行了消融研究,以隔离Sable性能提升的来源,并确认其高效的计算内存使用。

英文摘要:

As multi-agent reinforcement learning (MARL) progresses towards solving larger and more complex problems, it becomes increasingly important that algorithms exhibit the key properties of (1) strong performance, (2) memory efficiency, and (3) scalability. In this work, we introduce Sable, a performant, memory-efficient, and scalable sequence modeling approach to MARL. Sable works by adapting the retention mechanism in Retentive Networks (Sun et al., 2023) to achieve computationally efficient processing of multi-agent observations with long context memory for temporal reasoning. Through extensive evaluations across six diverse environments, we demonstrate how Sable is able to significantly outperform existing state-of-the-art methods in a large number of diverse tasks (34 out of 45 tested). Furthermore, Sable maintains performance as we scale the number of agents, handling environments with more than a thousand agents while exhibiting a linear increase in memory usage. Finally, we conduct ablation studies to isolate the source of Sable's performance gains and confirm its efficient computational memory usage.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑