arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2502.01971cs.MA
收藏

自下而上的声誉通过多智能体强化学习促进合作

Bottom-Up Reputation Promotes Cooperation with Multi-Agent Reinforcement Learning

Tianyu Ren, Xuan Yao, Yang Li, Xiao-Jun Zeng

AI总结:

本文提出自下而上的声誉学习方法LR2,通过声誉奖励塑形和策略聚类,在不依赖预定义规范的情况下促进多智能体在空间社会困境中的合作。

摘要:

声誉是促进多智能体系统中合作的有力机制,因为智能体更倾向于与具有良好社会声誉的个体合作。现有的多智能体强化学习方法通常依赖预定义的社会规范来分配声誉,但当智能体持有私有的、独立的观点时,群体如何在评判上达成共识这一问题仍未解决。本文提出了一种新颖的自下而上的声誉学习方法——带声誉奖励的学习(Learning with Reputation Reward,LR2),旨在通过基于所分配声誉的奖励塑形来促进合作行为。我们的智能体架构包含一个困境策略和一个评估策略:困境策略通过考虑对邻居的影响来决定是否合作,评估策略则分配声誉以影响邻居的行为,同时优化自身目标。该方法仅利用局部观测和基于交互的奖励运行,不依赖集中式模块或预定义规范。研究结果表明,LR2 在各种空间社会困境场景中均具有有效性和适应性。有趣的是,我们发现 LR2 不仅通过自下而上声誉的奖励重塑来稳定和增强合作,还通过促进结构化群体中的策略聚类来创造有利于持续合作的环境。

英文摘要:

Reputation serves as a powerful mechanism for promoting cooperation in multi-agent systems, as agents are more inclined to cooperate with those of good social standing. While existing multi-agent reinforcement learning methods typically rely on predefined social norms to assign reputations, the question of how a population reaches a consensus on judgement when agents hold private, independent views remains unresolved. In this paper, we propose a novel bottom-up reputation learning method, Learning with Reputation Reward (LR2), designed to promote cooperative behaviour through rewards shaping based on assigned reputation. Our agent architecture includes a dilemma policy that determines cooperation by considering the impact on neighbours, and an evaluation policy that assigns reputations to affect the actions of neighbours while optimizing self-objectives. It operates using local observations and interaction-based rewards, without relying on centralized modules or predefined norms. Our findings demonstrate the effectiveness and adaptability of LR2 across various spatial social dilemma scenarios. Interestingly, we find that LR2 stabilizes and enhances cooperation not only with reward reshaping from bottom-up reputation but also by fostering strategy clustering in structured populations, thereby creating environments conducive to sustained cooperation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑