分散式强化学习：通过局部经济交易进行全球决策
Decentralized Reinforcement Learning: Global Decision-Making via Local Economic Transactions
摘要：
本文旨在建立一个框架，指导一个简单的，专门的，自利的代理人的社会，解决传统上被称为单片单代理顺序决策问题。使用分散的方法来集体优化一个中心目标的挑战在于难以描述非合作博弈的均衡策略。为了克服这一挑战，我们设计了一种机制，用于定义每个代理的学习环境，我们知道，全球目标的最佳解决方案与代理优化自己的本地目标的纳什均衡策略配置文件相吻合。社会作为一个经济的代理人，学习信用分配过程本身通过购买和出售对环境状态的经营权。我们得到了一类分散的强化学习算法，不仅广泛适用于标准的强化学习，但也选择半MDP的选项和动态组成计算图。最后，我们展示了一个社会的内在模块化结构的潜在优势，更有效的迁移学习。
英文摘要：
This paper seeks to establish a framework for directing a society of simple, specialized, self-interested agents to solve what traditionally are posed as monolithic single-agent sequential decision problems. What makes it challenging to use a decentralized approach to collectively optimize a central objective is the difficulty in characterizing the equilibrium strategy profile of non-cooperative games. To overcome this challenge, we design a mechanism for defining the learning environment of each agent for which we know that the optimal solution for the global objective coincides with a Nash equilibrium strategy profile of the agents optimizing their own local objectives. The society functions as an economy of agents that learn the credit assignment process itself by buying and selling to each other the right to operate on the environment state. We derive a class of decentralized reinforcement learning algorithms that are broadly applicable not only to standard reinforcement learning but also for selecting options in semi-MDPs and dynamically composing computation graphs. Lastly, we demonstrate the potential advantages of a society's inherent modular structure for more efficient transfer learning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
