Being able to harness the power of large datasets for developing cooperative multi-agent controllers promises to unlock enormous value for real-world applications. Many important industrial systems are multi-agent in nature and are difficult to model using bespoke simulators. However, in industry, distributed processes can often be recorded during operation, and large quantities of demonstrative data stored. Offline multi-agent reinforcement learning (MARL) provides a promising paradigm for building effective decentralised controllers from such datasets. However, offline MARL is still in its infancy and therefore lacks standardised benchmark datasets and baselines typically found in more mature subfields of reinforcement learning (RL). These deficiencies make it difficult for the community to sensibly measure progress. In this work, we aim to fill this gap by releasing off-the-grid MARL (OG-MARL): a growing repository of high-quality datasets with baselines for cooperative offline MARL research. Our datasets provide settings that are characteristic of real-world systems, including complex environment dynamics, heterogeneous agents, non-stationarity, many agents, partial observability, suboptimality, sparse rewards and demonstrated coordination. For each setting, we provide a range of different dataset types (e.g. Good, Medium, Poor, and Replay) and profile the composition of experiences for each dataset. We hope that OG-MARL will serve the community as a reliable source of datasets and help drive progress, while also providing an accessible entry point for researchers new to the field.
Offline Pre-trained Multi-Agent Decision Transformer: One Big Sequence Model Tackles All SMAC Tasks
Institute of Automation, CAS（中国科学院自动化研究所）；School of Artificial Intelligence, UCAS（中国科学院大学人工智能学院）；Shanghai Jiao Tong University（上海交通大学）；King’s College London（伦敦国王学院）；University College London（伦敦大学学院）
Sim2O: Efficient Offline-to-Online MARL via Joint Action Composition
Department of Computer Science and Engineering, Washington University in St. Louis（圣路易斯华盛顿大学计算机科学与工程系）；The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China（中国科学院自动化研究所复杂系统认知与决策智能重点实验室）
A Model-Based Solution to the Offline Multi-Agent Reinforcement Learning Coordination Problem
Mila - Quebec AI Institute（魁北克人工智能研究所）；McGill University（麦吉尔大学）；University of Oxford（牛津大学）；The University of Texas at Austin（德克萨斯大学奥斯汀分校）；Meta AI - FAIR（Meta AI 基础人工智能研究团队）
MACCA: Offline Multi-agent Reinforcement Learning with Causal Credit Assignment
King’s College London（伦敦国王学院）；Eindhoven University of Technology（埃因霍温理工大学）；University of Liverpool（利物浦大学）；University of California San Diego（加州大学圣地亚哥分校）
Conservative and Risk-Aware Offline Multi-Agent Reinforcement Learning
Centre for Wireless Communications (CWC), University of Oulu（奥卢大学无线通信中心）；King’s College London（伦敦国王学院）；King Abdullah University of Science and Technology (KAUST)（阿卜杜拉国王科技大学）
Offline Multi-Agent Reinforcement Learning with Implicit Global-to-Local Value Regularization
Institute for AI Industry Research (AIR), Tsinghua University（清华大学智能产业研究院（AIR））；Beijing Jiaotong University（北京交通大学）；UT Austin（德克萨斯大学奥斯汀分校）；Tsinghua University（清华大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）
Offline Multi-agent Reinforcement Learning via Sequential Score Decomposition
School of Data Science, The Chinese University of Hong Kong, Shenzhen, China（香港中文大学（深圳）数据科学学院）；School of Computer Science, Tongji University, Shanghai, China（同济大学计算机科学学院）；Vector Institute（向量研究所）
Learning from Good Trajectories in Offline Multi-Agent Reinforcement Learning
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；School of Data Science, Chinese University of Hong Kong (Shenzhen)（香港中文大学（深圳）数据科学学院）
Sample-Efficient Tabular Self-Play for Offline Robust Reinforcement Learning
Zhejiang University（浙江大学）；The Chinese University of Hong Kong（香港中文大学）；Edith Cowan University（埃迪斯·科温大学）；University of New South Wales（新南威尔士大学）；Huazhong University of Science and Technology（华中科技大学）
Improving Generalization and Data Efficiency with Diffusion in Offline Multi-agent RL
Institute for Interdisciplinary Information Sciences（交叉信息学院）；Tsinghua University（清华大学）；Department of Electronic and Computer Engineering（电子与计算机工程系）；Hong Kong University of Science and Technology（香港科学与技术大学）
Contextual Transformer for Offline Meta Reinforcement Learning
University of Chinese Academy of Sciences（中国科学院大学）；Peking University（北京大学）；University College London（伦敦大学学院）；Chinese Academy of Sciences（中国科学院）；King’s College London（伦敦国王学院）
MARLEM: A Multi-Agent Reinforcement Learning Simulation Framework for Implicit Cooperation in Decentralized Local Energy Markets
Department of Mechanical Engineering, Universidad de los Andes（机械工程系，洛斯安第斯大学）；Department of Industrial Engineering, Universidad de los Andes（工业工程系，洛斯安第斯大学）
