发表机构
Université de Lorraine; CNRS; Inria; LORIA（洛林大学; 法国国家科学研究中心; 法国国家信息与自动化研究所; 洛林计算机科学与应用实验室）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究证明，在中等规模博弈中，简单的近似值迭代（AVI）自我对弈方法能学习到比AlphaZero更准确的价值函数，且其贪心策略在更低成本下与MCTS策略竞争力相当，挑战了复杂搜索方法的必要性。
AI 中文摘要
将搜索与函数逼近相结合推动了博弈程序领域的重大进展，使得自我对弈算法比以往任何时候都更具竞争力。然而，基于蒙特卡洛树搜索（MCTS）的最流行方法，其计算开销可能相当可观。在本研究中，我们探讨了在诸如四子棋、六边形棋（7x7）和合成博弈等非平凡的中等规模博弈中，更简单的方法是否仍能保持竞争力。我们训练了一个最小化的近似值迭代（AVI）自我对弈实现，并使用真实标签预言机进行精确评估。与预期相反，我们的结果展示了AVI的惊人有效性：它学习到的价值函数比AlphaZero学习到的更为准确，而其单步前瞻贪心策略在训练和推理成本大幅降低的情况下，仍能与基于MCTS的策略保持竞争力。在黑白棋和九路围棋上的初步实验表明，AVI在更大规模的博弈中训练稳定，并能学习到有效的价值函数。这些发现表明，基于MCTS的方法的成功可能掩盖了那些随着现代深度学习工具而日益实用的更简单方法。
英文摘要
Combining search with function approximation has driven major advances in game-playing programs, making self-play algorithms more competitive than ever. Still, the computational overhead of the most popular methods, based on Monte Carlo Tree Search (MCTS), can be substantial. In this work, we investigate whether simpler methods remain competitive in non-trivial, moderately sized games such as Connect Four, Hex(7x7) and synthetic games. We train a minimal self-play implementation of Approximate Value Iteration (AVI) and use ground-truth oracles for exact evaluation. Contrary to expectations, our results demonstrate the surprising effectiveness of AVI: it learns more accurate value functions than those learned by AlphaZero, while its one-step-lookahead greedy policies remain competitive with MCTS-based policies at substantially lower training and inference costs. Preliminary experiments on Othello and Go(9x9) show that AVI trains stably on larger games and learns effective value functions. These findings suggest that the success of MCTS-based methods may have eclipsed simpler approaches that have become increasingly practical with modern deep-learning tools.
