arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2107.01460cs.LGcs.MA
收藏

Mava:基于JAX的分布式多智能体强化学习研究库

Mava: a research library for distributed multi-agent reinforcement learning in JAX

InstaDeep(InstaDeep公司)

Ruan de Kock, Omayma Mahjoub, Sasha Abramowitz, Wiem Khlifi, Callum Rhys Tilbury, Claude Formanek, Andries Smit, Arnu Pretorius

AI总结:

本研究推出基于JAX的纯代码多智能体强化学习研究库Mava,兼具易用性与可扩展性,速度较主流框架提升10-100倍,可大幅缩短实验周期,助力MARL领域的可靠研究。

摘要:

多智能体强化学习(MARL)研究本质上计算成本高昂,通常难以获得足够数量的实验样本以验证假设并得出可靠的统计结论。此外,MARL算法的设计通常较为复杂,正确实现的难度较大。这些特性使得构建适用于前沿研究的实用软件面临巨大挑战。我们对此类软件的评判标准是:它应当足够简单易用,能够快速实现新想法,同时具备足够的可扩展性与速度,可在合理时间内完成想法的测试。在这份初步技术报告中,我们介绍了Mava——一个完全基于JAX编写的MARL研究库,旨在满足上述标准。我们探讨了Mava的设计与核心特性,并在多种环境中展示了其用法与性能。我们特别证明了Mava具备显著的速度优势:与其他主流MARL框架相比,性能提升达10至100倍,同时保持了优异的效果。这使得研究人员仅需数分钟而非数小时即可完成想法的测试。最后,Mava是一个可无缝集成的库生态系统的组成部分,该生态系统旨在为MARL领域的前沿研究提供支持。我们希望Mava能够惠及学界,推动该领域具备科学严谨性与统计可靠性的研究发展。Mava的开源代码仓库可访问https://github.com/instadeepai/Mava获取。

英文摘要:

Multi-agent reinforcement learning (MARL) research is inherently computationally expensive and it is often difficult to obtain a sufficient number of experiment samples to test hypotheses and make robust statistical claims. Furthermore, MARL algorithms are typically complex in their design and can be tricky to implement correctly. These aspects of MARL present a difficult challenge when it comes to creating useful software for advanced research. Our criteria for such software is that it should be simple enough to use to implement new ideas quickly, while at the same time be scalable and fast enough to test those ideas in a reasonable amount of time. In this preliminary technical report, we introduce Mava, a research library for MARL written purely in JAX, that aims to fulfill these criteria. We discuss the design and core features of Mava, and demonstrate its use and performance across a variety of environments. In particular, we show Mava's substantial speed advantage, with improvements of 10-100x compared to other popular MARL frameworks, while maintaining strong performance. This allows for researchers to test ideas in a few minutes instead of several hours. Finally, Mava forms part of an ecosystem of libraries that seamlessly integrate with each other to help facilitate advanced research in MARL. We hope Mava will benefit the community and help drive scientifically sound and statistically robust research in the field. The open-source repository for Mava is available at https://github.com/instadeepai/Mava.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑