This paper investigates the model-based methods in multi-agent reinforcement learning (MARL). We specify the dynamics sample complexity and the opponent sample complexity in MARL, and conduct a theoretic analysis of return discrepancy upper bound. To reduce the upper bound with the intention of low sample complexity during the whole learning process, we propose a novel decentralized model-based MARL method, named Adaptive Opponent-wise Rollout Policy Optimization (AORPO). In AORPO, each agent builds its multi-agent environment model, consisting of a dynamics model and multiple opponent models, and trains its policy with the adaptive opponent-wise rollout. We further prove the theoretic convergence of AORPO under reasonable assumptions. Empirical experiments on competitive and cooperative tasks demonstrate that AORPO can achieve improved sample efficiency with comparable asymptotic performance over the compared MARL methods.
SERA: Scale-Equalized Rollout Allocation for Maximum Likelihood Reinforcement Learning
Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Nanyang Technological University（南洋理工大学）
Efficient Multi-turn RL for GUI Agents via Decoupled Training and Adaptive Data Curation
Beijing Institute of Technology（北京理工大学）；State Key Laboratory of General Artificial Intelligence, BIGAI（通用人工智能国家重点实验室，BIGAI）；DataCanvas；Beijing University of Posts and Telecommunications（北京邮电大学）；Shenzhen MSU-BIT University（深圳MSU-BIT大学）
Learn What's Left, Not What's Mastered: Saturation Aware Advantage Reweighting for Multi-Reward Policy Optimization
University of Florida（佛罗里达大学）；UC San Diego（加州大学圣迭戈分校）；Northeastern University（东北大学）；Northwestern University（西北大学）；Stanford University（斯坦福大学）；Universität Innsbruck（因斯布鲁克大学）
Make Sparse Rewards Count: Density-Aware Reward Aggregation for Multi-Reward RL
University of Chinese Academy of Sciences（中国科学院大学）；University of Minnesota Twin Cities（明尼苏达大学双城分校）；University of Wisconsin–Madison（威斯康星大学麦迪逊分校）；Stony Brook University（纽约州立大学石溪分校）；Dalian University of Technology（大连理工大学）；Beijing Foreign Studies University（北京外国语大学）；Southeast University（东南大学）；Kuaishou Technology（快手科技）
3SPO: State-Score-Supervised Policy Optimization for LLM Agents
School of Computer Science and Technology, East China Normal University（东华大学计算机科学与技术学院）；Fudan University（复旦大学）；KAUST（卡塔尔人工智能研究学院）；Fuzhou University（福州大学）
GD$^2$PO: Mitigating Multi-Reward Conflicts via Group-Dynamic reward-Decoupled Policy Optimization
Qwen Large Model Application Team, Alibaba（阿里巴巴通义千问大模型应用团队）；Renmin University of China（中国人民大学）；Peking University（北京大学）；ETH Zürich（苏黎世联邦理工学院）；University of Zurich（苏黎世大学）；The Chinese University of Hong Kong（香港中文大学）
GraphHCA: Closed-Form Hindsight Credit Assignment for Long-Horizon LLM Agents
Beihang University（北京航空航天大学）；Zhongguancun Academy（中关村学院）；Communication University of China（中国传媒大学）；Hangzhou Innovation Institute of Beihang University（北京航空航天大学杭州创新研究院）
AGCD: Agent-Guided Cross-Modal Decoding for Weather Forecasting
Sun Yat-sen University（中山大学）；The Chinese University of Hong Kong（香港中文大学）；Jiangxi Science and Technology Normal University（江西科技师范大学）；China Meteorological Administration（中国气象局）；Huawei Technologies Co., Ltd（华为技术有限公司）；Guangdong-Hong Kong-Macao Greater Bay Area Weather Research Center for Monitoring Warning and Forecasting（粤港澳大湾区气象监测预警与预报研究中心）；National Supercomputing Center in Shenzhen（深圳国家超算中心）；Tsinghua University（清华大学）
ToMPO: Training LLM Strategic Decision Making from a Multi-Agent Perspective
State Key Laboratory of General Artificial Intelligence, BIGAI（通用人工智能国家重点实验室，BIGAI）；Peking University（北京大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Tsinghua University（清华大学）
REOPD: Reliability-Adaptive Reward Extrapolation for On-Policy Distillation
Peking University（北京大学）；Southwest Jiaotong University（西南交通大学）；University of Science and Technology of China（中国科学技术大学）；School of Computer Science and Technology,University of Science and Technology of China（中国科学技术大学计算机科学与技术学院）；University of Electronic Science and Technology of China（电子科技大学）；School of Automation Engineering,University of Electronic Science and Technology of China（电子科技大学自动化工程学院）；Renmin University of China（中国人民大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Autonomous Driving, Shanghai Artificial Intelligence Laboratory（上海人工智能实验室自动驾驶研究中心）
