同时移动博弈中蒙特卡罗树搜索的汉南一致选择分析
Analysis of Hannan Consistent Selection for Monte Carlo Tree Search in Simultaneous Move Games
摘要：
汉南一致性，或没有外部遗憾，是游戏学习的一个关键概念。一个动作选择算法是汉南一致的（HC），如果它的性能最终选择最好的固定动作在事后。如果~零和范式对策中的两个局中人都使用~Hannan一致算法，则他们的平均行为收敛于~对策的~纳什均衡（NE）。一个类似的结果是已知的广泛形式的游戏，但〜播放的策略需要汉南一致的〜反事实的价值观，这往往是很难获得。我们研究的零和广泛的形式的游戏，同时移动，但在其他方面完美的信息。这些对策是范式对策的推广，是广义对策的一个特例。我们研究是否适用HC算法在这些游戏的每个决策点直接观察到的收益导致收敛到一个~纳什均衡。这个学习过程对应于一类蒙特卡罗树搜索算法，这是流行的玩冒险移动游戏，但没有任何已知的性能保证。我们表明，直接使用HC算法的~观察到的支付是不足以保证~收敛。通过对联合动作进行~额外的平均，保证了~收敛，但经验上较慢。我们进一步定义了HC算法的一个额外的属性，这是足以保证的收敛性没有平均，我们经验表明，常用的HC算法有这个属性。
英文摘要：
Hannan consistency, or no external regret, is a~key concept for learning in games. An action selection algorithm is Hannan consistent (HC) if its performance is eventually as good as selecting the~best fixed action in hindsight. If both players in a~zero-sum normal form game use a~Hannan consistent algorithm, their average behavior converges to a~Nash equilibrium (NE) of the~game. A similar result is known about extensive form games, but the~played strategies need to be Hannan consistent with respect to the~counterfactual values, which are often difficult to obtain. We study zero-sum extensive form games with simultaneous moves, but otherwise perfect information. These games generalize normal form games and they are a special case of extensive form games. We study whether applying HC algorithms in each decision point of these games directly to the~observed payoffs leads to convergence to a~Nash equilibrium. This learning process corresponds to a~class of Monte Carlo Tree Search algorithms, which are popular for playing simultaneous-move games but do not have any known performance guarantees. We show that using HC algorithms directly on the~observed payoffs is not sufficient to guarantee the~convergence. With an~additional averaging over joint actions, the~convergence is guaranteed, but empirically slower. We further define an~additional property of HC algorithms, which is sufficient to guarantee the~convergence without the~averaging and we empirically show that commonly used HC algorithms have this property.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
