arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.02148cs.GTcs.AIcs.MA
收藏

序贯社会困境中的环境复杂度与纳什均衡

Environment Complexity and Nash Equilibria in a Sequential Social Dilemma

The Alan Turing Institute(艾伦·图灵研究所);University of Exeter(埃克塞特大学)

Mustafa Yasir, Andrew Howes, Vasilios Mavroudis, Chris Hicks

AI总结:

本文将矩阵博弈社会困境扩展为高维序贯 MARL 环境,通过调整猎鹿博弈网格世界并改变环境复杂度,发现复杂度升高会使智能体收敛到风险占优的次优纳什均衡策略,揭示了环境复杂度对合作最优结果的影响。

摘要:

多智能体强化学习(MARL)方法虽然在零和或正和博弈中有效,但在合作对于实现全局最优结果至关重要的一般和博弈中,往往会产生次优结果。矩阵博弈社会困境抽象了一般和交互的关键方面,如合作、风险和信任,但无法刻画现实世界场景所特有的时间和空间动态。为此,本研究将矩阵博弈社会困境扩展到更复杂、更高维的 MARL 环境中。我们对猎鹿博弈(Stag Hunt)的网格世界实现进行了调整,使其更接近一次性矩阵博弈的决策空间,同时引入可变的环境复杂度。研究结果表明,随着复杂度的增加,在这些环境中训练的 MARL 智能体会收敛到次优策略,这与矩阵博弈中风险占优的纳什均衡策略一致。我们的工作揭示了环境复杂度对在高维博弈论 MARL 环境中实现最优结果的影响。

英文摘要:

Multi-agent reinforcement learning (MARL) methods, while effective in zero-sum or positive-sum games, often yield suboptimal outcomes in general-sum games where cooperation is essential for achieving globally optimal outcomes. Matrix game social dilemmas, which abstract key aspects of general-sum interactions, such as cooperation, risk, and trust, fail to model the temporal and spatial dynamics characteristic of real-world scenarios. In response, our study extends matrix game social dilemmas into more complex, higher-dimensional MARL environments. We adapt a gridworld implementation of the Stag Hunt dilemma to more closely match the decision-space of a one-shot matrix game while also introducing variable environment complexity. Our findings indicate that as complexity increases, MARL agents trained in these environments converge to suboptimal strategies, consistent with the risk-dominant Nash equilibria strategies found in matrix games. Our work highlights the impact of environment complexity on achieving optimal outcomes in higher-dimensional game-theoretic MARL environments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑