Altruistic cooperation is costly yet socially desirable. As a result, agents struggle to learn cooperative policies through independent reinforcement learning (RL). Indirect reciprocity, where agents consider their interaction partner's reputation, has been shown to stabilise cooperation in homogeneous, idealised populations. However, more realistic settings are comprised of heterogeneous agents with different characteristics and group-based social identities. We study cooperation when agents are stratified into two such groups, and allow reputation updates and actions to depend on group information. We consider two modelling approaches: evolutionary game theory, where we comprehensively search for social norms (i.e., rules to assign reputations) leading to cooperation and fairness; and RL, where we consider how the stochastic dynamics of policy learning affects the analytically identified equilibria. We observe that a defecting majority leads the minority group to defect, but not the inverse. Moreover, changing the norms that judge in and out-group interactions can steer a system towards either fair or unfair cooperation. This is made clearer when moving beyond equilibrium analysis to independent RL agents, where convergence to fair cooperation occurs with a narrower set of norms. Our results highlight that, in heterogeneous populations with reputations, carefully defining interaction norms is fundamental to tackle both dilemmas of cooperation and of fairness.
Emergent Cooperation under Uncertain Incentive Alignment
University of Groningen（格罗宁根大学）；ILLC & IvI, University of Amsterdam（阿姆斯特丹大学逻辑、语言与计算研究所及信息学研究所）；Vrije Universiteit Brussels（布鲁塞尔自由大学）；Utrecht University（乌得勒支大学）
Talk, Judge, Cooperate: Gossip-Driven Indirect Reciprocity in Self-Interested LLM Agents
University of Waterloo（滑铁卢大学）；Vector Institute（Vector研究所）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Tongji University（同济大学）；Johns Hopkins University（约翰斯·霍普金斯大学）
Normative Disagreement as a Challenge for Cooperative AI
University of Tuebingen（图宾根大学）；Center on Long-Term Risk（长期风险中心）；New York University（纽约大学）；Vector Institute（矢量研究所）；University of Toronto（多伦多大学）；DeepMind
CoopEval: Benchmarking Cooperation-Sustaining Mechanisms and LLM Agents in Social Dilemmas
Carnegie Mellon University；Foundations of Cooperative AI Lab (FOCAL)；Jinesis Lab, University of Toronto \& Vector Institute；ETH Z\" u rich；Max Planck Institute for Intelligent Systems, T\" u bingen, Germany
