In the future, artificial learning agents are likely to become increasingly widespread in our society. They will interact with both other learning agents and humans in a variety of complex settings including social dilemmas. We consider the problem of how an external agent can promote cooperation between artificial learners by distributing additional rewards and punishments based on observing the learners' actions. We propose a rule for automatically learning how to create right incentives by considering the players' anticipated parameter updates. Using this learning rule leads to cooperation with high social welfare in matrix games in which the agents would otherwise learn to defect with high probability. We show that the resulting cooperative outcome is stable in certain games even if the planning agent is turned off after a given number of episodes, while other games require ongoing intervention to maintain mutual cooperation. However, even in the latter case, the amount of necessary additional incentives decreases over time.
Massachusetts Institute of Technology（麻省理工学院）；INESC-ID（里斯本大学高等理工学院附属信息系统与计算机工程研究所）；Instituto Superior Técnico, Universidade de Lisboa（里斯本大学高等理工学院）
Designing mechanisms that leverage cooperation between agents has been a long-lasting goal in Multiagent Systems. The task is especially challenging when agents are selfish, lack common goals and face social dilemmas, i.e., situations in which individual interest conflicts with social welfare. Past works explored mechanisms that explain cooperation in biological and social systems, providing important clues for the aim of designing cooperative artificial societies. In particular, several works show that cooperation is able to emerge when specific network structures underlie agents' interaction…
Groups of humans are often able to find ways to cooperate with one another in complex, temporally extended social dilemmas. Models based on behavioral economics are only able to explain this phenomenon for unrealistic stateless matrix games. Recently, multi-agent reinforcement learning has been applied to generalize social dilemma problems to temporally and spatially extended Markov games. However, this has not yet generated an agent that learns to cooperate in social dilemmas as humans do. A key insight is that many, but not all, human individuals have inequity averse social preferences. This…
University of California, Berkeley（加州大学伯克利分校）；Center for Human-Compatible AI（人类兼容人工智能中心）
Recent work on promoting cooperation in multi-agent learning has resulted in many methods which successfully promote cooperation at the cost of becoming more vulnerable to exploitation by malicious actors. We show that this is an unavoidable trade-off and propose an objective which balances these concerns, promoting both safety and long-term cooperation. Moreover, the trade-off between safety and cooperation is not severe, and you can receive exponentially large returns through cooperation from a small amount of risk. We study both an exact solution method and propose a method for training pol…
The Alan Turing Institute（艾伦·图灵研究所）；University College London（伦敦大学学院）；University of Bologna（博洛尼亚大学）
Social dilemmas have been widely studied to explain how humans are able to cooperate in society. Considerable effort has been invested in designing artificial agents for social dilemmas that incorporate explicit agent motivations that are chosen to favor coordinated or cooperative responses. The prevalence of this general approach points towards the importance of achieving an understanding of both an agent's internal design and external environment dynamics that facilitate cooperative behavior. In this paper, we investigate how partner selection can promote cooperative behavior between agents …
Multi-agent reinforcement learning (MARL) has achieved great progress in cooperative tasks in recent years. However, in the local reward scheme, where only local rewards for each agent are given without global rewards shared by all the agents, traditional MARL algorithms lack sufficient consideration of agents' mutual influence. In cooperative tasks, agents' mutual influence is especially important since agents are supposed to coordinate to achieve better performance. In this paper, we propose a novel algorithm Mutual-Help-based MARL (MH-MARL) to instruct agents to help each other in order to …
Universidad Distrital Francisco José de Caldas（弗朗西斯科·何塞·德·卡尔达斯地区大学）
With artificial intelligence systems becoming ubiquitous in our society, its designers will soon have to start to consider its social dimension, as many of these systems will have to interact among them to work efficiently. With this in mind, we propose a decentralized deep reinforcement learning algorithm for the design of cooperative multi-agent systems. The algorithm is based on the hypothesis that highly correlated actions are a feature of cooperative systems, and hence, we propose the insertion of an auxiliary objective of maximization of the mutual information between the actions of agen…
Evolutionary mechanisms that promote cooperation may not promote social welfare
Teesside University（提赛德大学）；University of Birmingham（伯明翰大学）；University of Maribor（马里博尔大学）；Community Healthcare Center Dr. Adolf Drolc Maribor（阿道夫·德罗尔奇博士马里博尔社区医疗中心）；Complexity Science Hub Vienna（维也纳复杂科学中心）；Kyung Hee University（庆熙大学）
Cooperative behavior in real social dilemmas is often perceived as a phenomenon emerging from norms and punishment. To overcome this paradigm, we highlight the interplay between the influence of social networks on individuals, and the activation of spontaneous self-regulating mechanisms, which may lead them to behave cooperatively, while interacting with others and taking conflicting decisions over time. By extending Evolutionary game theory over networks, we prove that cooperation partially or fully emerges whether self-regulating mechanisms are sufficiently stronger than social pressure. Int…
Enforcing cooperation among substantial agents is one of the main objectives for multi-agent systems. However, due to the existence of inherent social dilemmas in many scenarios, the free-rider problem may arise during agents' long-run interactions and things become even severer when self-interested agents work in collusion with each other to get extra benefits. It is commonly accepted that in such social dilemmas, there exists no simple strategy for an agent whereby she can simultaneously manipulate on the utility of each of her opponents and further promote mutual cooperation among all agent…
Prosocial incentive can promote cooperation, but providing incentive is costly. Institutions in human society may prefer to use an incentive strategy which is able to promote cooperation at a reasonable cost. However, thus far few works have explored the optimal institutional incentives which minimize related cost for the benefit of public cooperation. In this work, in combination with optimal control theory we thus formulate two optimal control problems to explore the optimal incentive strategies for institutional reward and punishment respectively. By using the approach of Hamilton-Jacobi-Be…
Individual migration has been regarded as an important factor for the evolution of cooperation in mobile populations. Motivations of migration, however, can be largely divergent: one is highly frustrated by the vicinity of an exploiter or defector, while other enthusiastically searches cooperator mates. Albeit both extreme attitudes are observed in human behavior, but their specific impacts on wellbeing remained unexplored. In this work, we propose an orientation-driven migration approach for mobile individuals in combination with the mentioned migration preferences and study their roles in th…
Modeling Decision-Making with Will for Cooperation in Social Dilemmas
School of Intelligence Science and Technology, Peking University（北京大学智能学院）；State Key Laboratory of General Artificial Intelligence, BIGAI（通用人工智能国家重点实验室，BIGAI）；Law School, Peking University（北京大学法学院）
