Recent reinforcement learning studies extensively explore the interplay between cooperative and competitive behaviour in mixed environments. Unlike cooperative environments where agents strive towards a common goal, mixed environments are notorious for the conflicts of selfish and social interests. As a consequence, purely rational agents often struggle to achieve and maintain cooperation. A prevalent approach to induce cooperative behaviour is to assign additional rewards based on other agents' well-being. However, this approach suffers from the issue of multi-agent credit assignment, which can hinder performance. This issue is efficiently alleviated in cooperative setting with such state-of-the-art algorithms as QMIX and COMA. Still, when applied to mixed environments, these algorithms may result in unfair allocation of rewards. We propose BAROCCO, an extension of these algorithms capable to balance individual and social incentives. The mechanism behind BAROCCO is to train two distinct but interwoven components that jointly affect each agent's decisions. Our meta-algorithm is compatible with both Q-learning and Actor-Critic frameworks. We experimentally confirm the advantages over the existing methods and explore the behavioural aspects of BAROCCO in two mixed multi-agent setups.
Balancing Efficiency and Fairness: An Iterative Exchange Framework for Multi-UAV Cooperative Path Planning
The Hong Kong University of Science and Technology(香港科技大学);College of Automation Engineering, Nanjing University of Aeronautics and Astronautics(南京航空航天大学自动化工程学院);College of Computer Science and Engineering, Chongqing Three Gorges University(重庆三峡大学计算机科学与工程学院);Chinese Academy of Sciences, New Territories, Hong Kong(中国科学院香港新 Territories 分院);City University of Hong Kong, Kowloon, Hong Kong(香港城市大学)
ACE: Cooperative Multi-agent Q-learning with Bidirectional Action-Dependency
The University of Sydney(悉尼大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);SenseTime Group LTD(商汤科技集团);Institute for AI, Peking University(北京大学人工智能研究院)
多智能体强化学习(MARL)存在非平稳性问题,即当多个智能体同时更新策略时,每一次迭代的目标都在不断变化。本文从第一性原理出发,提出了双向动作依赖Q学习(ACE)方法来解决非平稳性问题。ACE发展的核心是顺序决策过程,其中一次只允许一个代理采取行动。在这个过程中,每个代理最大化其价值函数给定的行动所采取的前代理在推理阶段。在学习阶段,每个代理最小化TD错误,这取决于后续代理如何对他们选择的动作做出反应。考虑到双向依赖的设计,ACE有效地将多Agent MDP转化为单Agent MDP。我们通过识别适当的网络表示来制定动作依赖关系,从而在一个向前传递中隐式地计算顺序决策过程,从而实现ACE框架。为了验证ACE,我们将其与两个MARL基准的强基线进行比较。在Google Research Football和StarCraft Multi-Agent Challenge上的实验结果表明,ACE算法的性能明显优于现有算法。特别是在SMAC任务上,ACE在几乎所有的硬地图和超硬地图上都达到了100%的成功率。我们进一步研究广泛的研究问题,ACE,包括扩展,推广,和实用性。提供代码以方便进一步研究。
CollabBench: Benchmarking and Unleashing Collaborative Ability of LLMs with Diverse Players via Proactive Engagement
Shanghai Institute of AI for Education(上海人工智能教育研究院);School of Computer Science(计算机科学学院);East China Normal University(东华大学);Tencent Inc.(腾讯公司);Shanghai Innovation Institute(上海创新研究院)