信息素启发的通信框架forLarge规模多智能体强化学习
PooL: Pheromone-inspired Communication Framework forLarge Scale Multi-Agent Reinforcement Learning
摘要：
难以扩展给多智能体协调带来了很大的问题。多智能体强化学习（MARL）算法在小规模多智能体系统中的应用很难扩展到大规模多智能体系统中，因为后者的动态性更强，交互次数随着智能体数量的增加而呈指数级增加。一些群体智能算法模拟信息素的释放和利用机制来控制大规模的Agent协调。受这些算法的启发，为了解决大规模多智能体协调问题，提出了一种基于\textbf{p}和\textbf{o} m \textbf{o} ne的间接通信框架\textbf{PooL}，用于大规模多智能体强化\textbf{l}学习. PooL的代理释放的信息素被定义为大多数强化学习算法的输出，它反映了代理对当前环境的看法。信息素更新机制可以有效地组织所有代理的信息，并将代理之间的复杂交互简化为低维表示。智能体感知到的信息素可以看作是附近智能体观点的总结，更能反映环境的真实情况。以Q-Learning为基础模型实现PooL，并在各种大规模协作环境中对PooL进行了评估。实验结果表明，通过PooL，Agent可以捕获有效的信息，并以较低的通信代价获得比其他方法更高的回报。
英文摘要：
Being difficult to scale poses great problems in multi-agent coordination. Multi-agent Reinforcement Learning (MARL) algorithms applied in small-scale multi-agent systems are hard to extend to large-scale ones because the latter is far more dynamic and the number of interactions increases exponentially with the growing number of agents. Some swarm intelligence algorithms simulate the release and utilization mechanism of pheromones to control large-scale agent coordination. Inspired by such algorithms, \textbf{PooL}, an \textbf{p}her\textbf{o}m\textbf{o}ne-based indirect communication framework applied to large scale multi-agent reinforcement \textbf{l}earning is proposed in order to solve the large-scale multi-agent coordination problem. Pheromones released by agents of PooL are defined as outputs of most reinforcement learning algorithms, which reflect agents' views of the current environment. The pheromone update mechanism can efficiently organize the information of all agents and simplify the complex interactions among agents into low-dimensional representations. Pheromones perceived by agents can be regarded as a summary of the views of nearby agents which can better reflect the real situation of the environment. Q-Learning is taken as our base model to implement PooL and PooL is evaluated in various large-scale cooperative environments. Experiments show agents can capture effective information through PooL and achieve higher rewards than other state-of-arts methods with lower communication costs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
