arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2206.03348cs.GTcs.AIcs.LG
收藏

具有高社会福利的Nash均衡的规范引导学习

Specification-Guided Learning of Nash Equilibria with High Social Welfare

University of Pennsylvania(宾夕法尼亚大学)

Kishor Jothimurugan, Suguman Bansal, Osbert Bastani, Rajeev Alur

摘要:

强化学习已被证明是一种有效的策略,自动训练具有挑战性的控制问题的政策。针对非合作多智能体系统,我们提出了一种新的强化学习框架,用于训练形成纳什均衡的联合策略。在我们的方法中,而不是提供低级别的奖励功能,用户提供高级别的规格,编码每个代理的目标。然后,在规范结构的指导下,我们的算法搜索策略,以确定可证明形成$ε$ -纳什均衡的策略(概率很高)。重要的是,它以最大化所有行为体的社会福利的方式确定政策的优先次序。我们的实证评估表明,我们的算法计算的均衡政策与高社会福利,而国家的最先进的基线要么无法计算纳什均衡或计算的社会福利相对较低。

英文摘要:

Reinforcement learning has been shown to be an effective strategy for automatically training policies for challenging control problems. Focusing on non-cooperative multi-agent systems, we propose a novel reinforcement learning framework for training joint policies that form a Nash equilibrium. In our approach, rather than providing low-level reward functions, the user provides high-level specifications that encode the objective of each agent. Then, guided by the structure of the specifications, our algorithm searches over policies to identify one that provably forms an $ε$-Nash equilibrium (with high probability). Importantly, it prioritizes policies in a way that maximizes social welfare across all agents. Our empirical evaluation demonstrates that our algorithm computes equilibrium policies with high social welfare, whereas state-of-the-art baselines either fail to compute Nash equilibria or compute ones with comparatively lower social welfare.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑