We present a multi-agent learning algorithm, ALMA-Learning, for efficient and fair allocations in large-scale systems. We circumvent the traditional pitfalls of multi-agent learning (e.g., the moving target problem, the curse of dimensionality, or the need for mutually consistent actions) by relying on the ALMA heuristic as a coordination mechanism for each stage game. ALMA-Learning is decentralized, observes only own action/reward pairs, requires no inter-agent communication, and achieves near-optimal (<5% loss) and fair coordination in a variety of synthetic scenarios and a real-world meeting scheduling problem. The lightweight nature and fast learning constitute ALMA-Learning ideal for on-device deployment.
On Improving Model-Free Algorithms for Decentralized Multi-Agent Reinforcement Learning
University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of California, Los Angeles(加利福尼亚大学洛杉矶分校);DeepMind(深度思维);Massachusetts Institute of Technology(麻省理工学院)
多智能体强化学习(MARL)算法经常遭受指数样本复杂度对智能体数量的依赖,这种现象被称为\emph{the curse of multiagents}。在本文中,我们解决这一挑战,调查样本效率的无模型算法在\emph{decentralized} MARL,并旨在改善现有的算法沿这条线。对于学习(粗)一般和马尔可夫博弈中的相关均衡,我们提出了\emph{stage-based} V-学习算法,大大简化了最近的作品的算法设计和分析,并规避了一个相当复杂的无\emph{weighted}遗憾强盗子程序。针对马尔可夫势博弈中纳什均衡的学习问题,提出了一种独立的策略梯度算法,该算法采用分散的基于动量的方差缩减技术。我们所有的算法都是去中心化的,因为每个代理只能根据其本地信息做出决策。在学习过程中既不需要通信,也不需要集中协调,从而导致对大量代理的自然泛化。我们还提供了数值模拟来证实我们的理论研究结果。
Improving International Climate Policy via Mutually Conditional Binding Commitments
Potsdam Institute for Climate Impact Research(波茨坦气候影响研究所);Alfred Weber Institute for Economics(阿尔弗雷德·韦伯经济研究所);University of Heidelberg(海德堡大学);Technical University of Berlin(柏林工业大学);Dyson Robot Learning Lab(戴森机器人学习实验室)
Improving Supply Chain Coordination by Linking Dynamic Procurement Decision to Multi-Agent System
互联网在许多方面改变了商业运作的方式。例如,在采购领域,与贸易伙伴直接互动的可能性导致了供应链管理方面的新机制。一种这样的交互式动态采购,它允许买方和卖方软件代理由潜在的买方代理投标,而不是由供应商进行静态采购。动态采购决策可以为买方提供买卖渠道,避免卖方不能履行合同承诺的情况发生。以NYOP(Name Your Own Price)为核心的动态采购协商算法建立了多Agent动态供应链系统,用JADE表示动态供应链信息网络DSIN(Dynamic Supply Chain Information Networks),用eM-Plant表示动态供应链物流仿真。最后,以供应链绩效指标(如牛鞭效应、成交率)来评估供应链绩效,作为企业未来决策的参考。
HULK: Large-scale Hierarchical Coordination under Continual and Uncertain Temporal Tasks
Department of Mechanics and Engineering Science, College of Engineering, Peking University(力学与工程科学系,工程学院,北京大学);National University of Defense Technology(国防科学技术大学)
Optimizing Highway Traffic Flow in Mixed Autonomy: A Multiagent Truncated Rollout Approach
School of Transportation, Tongji University(同济大学交通运输学院);Key Laboratory of Road and Traffic Engineering of Ministry of Education, Tongji University(同济大学道路与交通工程教育部重点实验室)