基于树复形上Blackwell可达性的扩展式博弈求解
Extensive-Form Game Solving via Blackwell Approachability on Treeplexes
AI总结:
该研究提出首个序列形式多面体上的Blackwell可达性算法框架,衍生步长不变的后悔最小化算法,结合预测在线镜像下降提出PTB+并优化得到O(1/T)收敛速率的算法,实验验证了其性能优势。
摘要:
本文提出了首个在序列形式多面体(一类刻画扩展式博弈(EFG)中玩家策略的凸多面体)上实现Blackwell可达性的算法框架。由此衍生出一类步长不变的新型后悔最小化算法,其步长不变特性与单纯形上的Regret Matching(后悔匹配)及Regret Matching$^+$算法一致。我们的模块化框架可与任意现有的锥上后悔最小化器结合,通过自博弈框架求解完美回忆下两人零和扩展式博弈的纳什均衡。借助预测在线镜像下降,我们提出了Predictive Treeplex Blackwell$^+$(PTB$^+$)算法,并证明其在自博弈中收敛到纳什均衡的速率为$O(1/\bar{T})$。随后我们展示了如何通过步长稳定PTB$^+$,得到具有当前最优$O(1/T)$收敛速率的算法。我们开展了大量实验,将该框架与CFR$^+$及其预测变体等多个算法基准进行对比,并揭示了实际性能与经典算法的步长依赖或步长不变特性之间的有趣关联。
英文摘要:
In this paper, we introduce the first algorithmic framework for Blackwell approachability on the sequence-form polytope, the class of convex polytopes capturing the strategies of players in extensive-form games (EFGs). This leads to a new class of regret-minimization algorithms that are stepsize-invariant, in the same sense as the Regret Matching and Regret Matching$^+$ algorithms for the simplex. Our modular framework can be combined with any existing regret minimizer over cones to compute a Nash equilibrium in two-player zero-sum EFGs with perfect recall, through the self-play framework. Leveraging predictive online mirror descent, we introduce Predictive Treeplex Blackwell$^+$ (PTB$^+$), and show a $O(1/\sqrt{T})$ convergence rate to Nash equilibrium in self-play. We then show how to stabilize PTB$^+$ with a stepsize, resulting in an algorithm with a state-of-the-art $O(1/T)$ convergence rate. We provide an extensive set of experiments to compare our framework with several algorithmic benchmarks, including CFR$^+$ and its predictive variant, and we highlight interesting connections between practical performance and the stepsize-dependence or stepsize-invariance properties of classical algorithms.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。