Curriculum Learning With Counterfactual Group Relative Policy Advantage For Multi-Agent Reinforcement Learning
School of Information and Communications Engineering, Xi'an Jiaotong University(西安交通大学信息与通信工程学院);Department of Computer Science and Engineering, The Chinese University of Hong Kong(香港中文大学计算机科学与工程系);School of Computer Science and Technology, University of Science and Technology of China(中国科学技术大学计算机科学与技术学院);Department of Electrical and Electronic Engineering, The University of Hong Kong(香港大学电子与电气工程系)
Weiqiang Jin, Hongyang Du, Guizhong Liu, Dong In Kim
Multi-agent reinforcement learning (MARL) has achieved strong performance in cooperative adversarial tasks. However, most existing methods typically train agents against fixed opponent strategies and rely on such meta-static difficulty conditions, which limits their adaptability to changing environments and often leads to suboptimal policies. Inspired by the success of curriculum learning (CL) in supervised tasks, we propose a dynamic CL framework for MARL that employs an self-adaptive difficulty adjustment mechanism. This mechanism continuously modulates opponent strength based on real-time agent training performance, allowing agents to progressively learn from easier to more challenging scenarios. However, the dynamic nature of CL introduces instability due to nonstationary environments and sparse global rewards. To address this challenge, we develop a Counterfactual Group Relative Policy Advantage (CGRPA), which is tightly coupled with the curriculum by providing intrinsic credit signals that reflect each agent's impact under evolving task demands. CGRPA constructs a counterfactual advantage function that isolates individual contributions within group behavior, facilitating more reliable policy updates throughout the curriculum. CGRPA evaluates each agent's contribution through constructing counterfactual action advantage function, providing intrinsic rewards that enhance credit assignment and stabilize learning under non-stationary conditions. Extensive experiments demonstrate that our method improves both training stability and final performance, achieving competitive results against state-of-the-art methods. The code is available at https://github.com/NICE-HKU/CL2MARL-SMAC.
UC-PSRO: Utility-Conditioned Policy-Space Response Oracles with a Communication-Dropout Curriculum for Game-Theoretic Course-of-Action Generation in Adversarial Swarms
Trajectory-Level Automatic Curriculum Learning for Legged Locomotion on Unstructured Terrain
National Key Laboratory of General Artificial Intelligence, BIGAI(通用人工智能国家重点实验室(BIGAI));Beijing Normal University(北京师范大学);Communication University of China(中国传媒大学);Shanghai Jiao Tong University(上海交通大学)
Reinforcement Learning with Curriculum-inspired Adaptive Direct Policy Guidance for Truck Dispatching
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院)
Curriculum-RLAIF: Curriculum Alignment with Reinforcement Learning from AI Feedback
Tsinghua University(清华大学);Provable Responsible AI and Data Analytics Lab(可证明责任AI与数据 analytics 实验室);King Abdullah University of Science and Technology(卡布斯王国科学与技术大学);University of Hamburg(汉堡大学);Tencent Inc.(腾讯公司);Shanghai Jiao Tong University(上海交通大学)
Learning Multi-Objective Curricula for Robotic Policy Learning
McGill University(麦吉尔大学);Mila - Québec AI Institute(米拉-魁北克人工智能研究所);IBM Research(IBM研究院);Université de Montréal(蒙特利尔大学);Polytechnique Montréal(蒙特利尔理工学院);Beijing Academy of AI(北京人工智能研究院)
Actor-Curator: Co-adaptive Curriculum Learning via Policy-Improvement Bandits for RL Post-Training
University of Illinois Chicago(伊利诺伊大学香槟分校);Caltech(加州理工学院);RPI(罗切斯特理工学院);MBZUAI(澳门大学人工智能研究院);University of Chicago(芝加哥大学);NEC Laboratories America(NEC美国实验室)
Curriculum Group Policy Optimization: Adaptive Sampling for Unleashing the Potential of Text-to-Image Generation
School of Artificial Intelligence, Beijing University of Posts and Telecommunications(北京邮电大学人工智能学院);Institute of Artificial Intelligence (TeleAI), China Telecom(中国电信人工智能研究院);Beijing Key Laboratory of Multimodal Data Intelligent Perception and Governance(北京多模态数据智能感知与治理重点实验室)
Wuhan University of Science and Technology(武汉科技大学);College of Science(理学院);Intelligent Automobile Engineering Research Institute(智能汽车工程研究院);School of Automobile and Traffic Engineering(汽车与交通工程学院)
On-Policy Distillation with Curriculum Turn-level Guidance for Multi-turn Agents
Foundation Model Research Center, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所基础模型研究中心);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Large Language Model Department, Tencent(腾讯大语言模型部);University of Science and Technology of China(中国科学技术大学);Zhejiang University(浙江大学);National University of Singapore(新加坡国立大学);Wuhan AI Research(武汉人工智能研究院)
Budgeted Multi-Source Counterfactual Annotation for Off-Policy Evaluation
Johns Hopkins Carey Business School(约翰斯·霍普金斯凯瑞商学院);International Computer Science Institute(国际计算机科学研究所);Pennsylvania State University(宾夕法尼亚州立大学);Georgia Institute of Technology(佐治亚理工学院)