Multi-agent reinforcement learning (MARL) research is inherently computationally expensive and it is often difficult to obtain a sufficient number of experiment samples to test hypotheses and make robust statistical claims. Furthermore, MARL algorithms are typically complex in their design and can be tricky to implement correctly. These aspects of MARL present a difficult challenge when it comes to creating useful software for advanced research. Our criteria for such software is that it should be simple enough to use to implement new ideas quickly, while at the same time be scalable and fast enough to test those ideas in a reasonable amount of time. In this preliminary technical report, we introduce Mava, a research library for MARL written purely in JAX, that aims to fulfill these criteria. We discuss the design and core features of Mava, and demonstrate its use and performance across a variety of environments. In particular, we show Mava's substantial speed advantage, with improvements of 10-100x compared to other popular MARL frameworks, while maintaining strong performance. This allows for researchers to test ideas in a few minutes instead of several hours. Finally, Mava forms part of an ecosystem of libraries that seamlessly integrate with each other to help facilitate advanced research in MARL. We hope Mava will benefit the community and help drive scientifically sound and statistically robust research in the field. The open-source repository for Mava is available at https://github.com/instadeepai/Mava.
JaxMARL: Multi-Agent RL Environments and Algorithms in JAX
University of Oxford(牛津大学);Universitat Politècnica de Catalunya(巴塞罗那理工大学);University College London(伦敦大学学院);University of Maryland(马里兰大学);Technical University Berlin(柏林技术大学)
EagerPy: Writing Code That Works Natively with PyTorch, TensorFlow, JAX, and NumPy
Tübingen AI Center, University of Tübingen(蒂宾根人工智能中心,蒂宾根大学);International Max Planck Research School for Intelligent Systems(国际马克斯·普朗克智能系统研究学院);Bernstein Center for Computational Neuroscience Tübingen(伯恩斯坦计算神经科学中心(蒂宾根))
Re-Envisioning Numerical Information Field Theory (NIFTy.re): A Library for Gaussian Processes and Variational Inference
Max Planck Institute for Astrophysics(马克斯·普朗克天体物理研究所);Ludwig Maximilian University of Munich(慕尼黑路德维希·马克西米利安大学);University of Vienna(维也纳大学);Technical University of Munich(慕尼黑工业大学);Helmholtz Zentrum München(慕尼黑亥姆霍兹中心)