In teacher-student framework, a more experienced agent (teacher) helps accelerate the learning of another agent (student) by suggesting actions to take in certain states. In cooperative multiagent reinforcement learning (MARL), where agents need to cooperate with one another, a student may fail to cooperate well with others even by following the teachers' suggested actions, as the polices of all agents are ever changing before convergence. When the number of times that agents communicate with one another is limited (i.e., there is budget constraint), the advising strategy that uses actions as advices may not be good enough. We propose a partaker-sharer advising framework (PSAF) for cooperative MARL agents learning with budget constraint. In PSAF, each Q-learner can decide when to ask for Q-values and share its Q-values. We perform experiments in three typical multiagent learning problems. Evaluation results show that our approach PSAF outperforms existing advising methods under both unlimited and limited budget, and we give an analysis of the impact of advising actions and sharing Q-values on agents' learning.
One of the key challenges for multi-agent learning is scalability. In this paper, we introduce a technique for speeding up multi-agent learning by exploiting concurrent and incremental experience sharing. This solution adaptively identifies opportunities to transfer experiences between agents and allows for the rapid acquisition of appropriate policies in large-scale, stochastic, homogeneous multi-agent systems. We introduce an online, distributed, supervisor-directed transfer technique for constructing high-level characterizations of an agent's dynamic learning environment---called contexts--…
The current paper deals with limited-budget output consensus for descriptor multiagent systems with two types of switching communication topologies; that is, switching connected ones and jointly connected ones. Firstly, a singular dynamic output feedback control protocol with switching communication topologies is proposed on the basis of the observable decomposition, where an energy constraint is involved and protocol states of neighboring agents are utilized to derive a new two-step design approach of gain matrices. Then, limited-budget output consensus problems are transformed into asymptoti…
College of Engineering, University of Georgia（佐治亚大学工程学院）；School of Computing, University of Georgia（佐治亚大学计算学院）
Achieving knowledge sharing within an artificial swarm system could lead to significant development in autonomous multiagent and robotic systems research and realize collective intelligence. However, this is difficult to achieve since there is no generic framework to transfer skills between agents other than a query-response-based approach. Moreover, natural living systems have a "forgetfulness" property for everything they learn. Analyzing such ephemeral nature (temporal memory properties of new knowledge gained) in artificial systems has never been studied in the literature. We propose a beh…
There are several real-world tasks that would benefit from applying multiagent reinforcement learning (MARL) algorithms, including the coordination among self-driving cars. The real world has challenging conditions for multiagent learning systems, such as its partial observable and nonstationary nature. Moreover, if agents must share a limited resource (e.g. network bandwidth) they must all learn how to coordinate resource use. This paper introduces a deep recurrent multiagent actor-critic framework (R-MADDPG) for handling multiagent coordination under partial observable set-tings and limited …
University Pompeu Fabra（庞培法布拉大学）；ICREA（加泰罗尼亚研究与高级研究中心）
There is a growing interest in Multi-Agent Reinforcement Learning (MARL) as the first steps towards building general intelligent agents that learn to make low and high-level decisions in non-stationary complex environments in the presence of other agents. Previous results point us towards increased conditions for coordination, efficiency/fairness, and common-pool resource sharing. We further study coordination in multi-task environments where several rewarding tasks can be performed and thus agents don't necessarily need to perform well in all tasks, but under certain conditions may specialize…
The use of game theoretic methods for control in multiagent systems has been an important topic in recent research. Valid utility games in particular have been used to model real-world problems; such games have the convenient property that the value of any decision set which is a Nash equilibrium of the game is guaranteed to be within 1/2 of the value of the optimal decision set. However, an implicit assumption in this guarantee is that each agent is aware of the decisions of all other agents. In this work, we first describe how this guarantee degrades as agents are only aware of a subset of t…
In this paper we explore how actor-critic methods in deep reinforcement learning, in particular Asynchronous Advantage Actor-Critic (A3C), can be extended with agent modeling. Inspired by recent works on representation learning and multiagent deep reinforcement learning, we propose two architectures to perform agent modeling: the first one based on parameter sharing, and the second one based on agent policy features. Both architectures aim to learn other agents' policies as auxiliary tasks, besides the standard actor (policy) and critic (values). We performed experiments in both cooperative an…
An Embodied Multiagent Framework Based on Token Communications for Cooperative ISAC
Tsinghua University（清华大学）；State Key Laboratory of Space Network and Communications, Tsinghua University（空间网络与通信国家重点实验室（清华大学））；Beijing National Research Center for Information Science and Technology, Tsinghua University（北京信息科学与技术国家研究中心（清华大学））
Multi-agent systems are designed to concurrently accomplish a diverse set of tasks at unprecedented scale. Here, the central problems faced by a system operator are to decide (i) how to divide available resources amongst the agents assigned to tasks and (ii) how to coordinate the behavior of the agents to optimize the efficiency of the resulting collective behavior. The focus of this paper is on problem (i), where we seek to characterize the impact of the division of resources on the best-case efficiency of the resulting collective behavior. Specifically, we focus on a team Colonel Blotto game…
Split over $n$ resource sharing problem: Are fewer capable agents better than many simpler ones?
MIST lab, École Polytechnique de Montréal, Montreal, Canada（蒙特利尔理工学院MIST实验室）；School of Electrical and Electronic Engineering, The University of Sheffield, Sheffield, UK（谢菲尔德大学电子与电气工程学院）；Department of Computer and Information Science, University of Konstanz, Konstanz, Germany（康斯坦茨大学计算机与信息科学系）；Department of Computer Science, Technical University of Darmstadt, Darmstadt, Germany（德累斯顿技术大学计算机科学系）
