通过基于策略的深度强化学习研究对手行为实现相关均衡
Achieving Correlated Equilibrium by Studying Opponent's Behavior Through Policy-Based Deep Reinforcement Learning
摘要：
博弈论是对分布式决策行为的一个非常深入的研究，并得到了许多学者的广泛发展。然而，许多现有的作品依赖于某些严格的假设，如知道对手的私人行为，这可能是不切实际的。在这项工作中，我们集中在两个诺贝尔奖的概念，纳什均衡和相关均衡。具体来说，我们成功地达到了纳什均衡的凸包外的相关均衡与我们提出的深度强化学习算法。利用相关均衡概率分布，我们还提出了一个数学模型来逆计算相关均衡概率分布，以估计对手的支付向量。有了这些收益，深度强化学习就能学习理性对手为什么以及如何比赛，而不仅仅是学习相应策略和行动的区域。仿真结果表明，在有限的参与者交互下，该方法能够获得最优的相关均衡，并在纳什均衡的凸包之外。
英文摘要：
Game theory is a very profound study on distributed decision-making behavior and has been extensively developed by many scholars. However, many existing works rely on certain strict assumptions such as knowing the opponent's private behaviors, which might not be practical. In this work, we focused on two Nobel winning concepts, the Nash equilibrium and the correlated equilibrium. Specifically, we successfully reached the correlated equilibrium outside the convex hull of the Nash equilibria with our proposed deep reinforcement learning algorithm. With the correlated equilibrium probability distribution, we also propose a mathematical model to inverse the calculation of the correlated equilibrium probability distribution to estimate the opponent's payoff vector. With those payoffs, deep reinforcement learning learns why and how the rational opponent plays, instead of just learning the regions for corresponding strategies and actions. Through simulations, we showed that our proposed method can achieve the optimal correlated equilibrium and outside the convex hull of the Nash equilibrium with limited interaction among players.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
