Tristan Cazenave
We present a general algorithm to order moves so as to speedup exact game solvers. It uses online learning of playout policies and Monte Carlo Tree Search. The learned policy and the information in the Monte Carlo tree are used to order moves in game solvers. They improve greatly the solving time for multiple games.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
继续探索
Artificial Intelligence is becoming instrumental in a variety of applications. Games serve as a good breeding ground for trying and testing these algorithms in a sandbox with simpler constraints in comparison to real life. In this project, we aim to develop an AI agent that can solve the classical Japanese game of Sokoban using various algorithms and heuristics and compare their performances through standard metrics.
On the Geometry of Games and their Solvers
提出一种结构感知的求解器合成框架，通过学习连续求解器对齐的博弈几何表示，实现自适应均衡计算并揭示求解器行为的连续区域。
Game Theory studies situations in which multiple agents having conflicting objectives have to reach a collective decision. The question of a compact representation language for agents utility function is of crucial importance since the classical representation of a $n$-players game is given by a $n$-dimensional matrix of exponential size for each player. In this paper we use the framework of Constraint Games in which CSP are used to represent utilities. Constraint Programming --including global constraints-- allows to easily give a compact and elegant model to many useful games. Constraint Gam…
Mitigating Legibility Tax with Decoupled Prover-Verifier Games
提出解耦证明者-验证者游戏（DPVG），通过分离正确性与可检查性训练一个翻译器模型，将固定求解器的解转化为可检查形式，在保持答案正确性的同时提高可检查性，解决了可读性代价问题。
Kingdomino is introduced as an interesting game for studying game playing: the game is multiplayer (4 independent players per game); it has a limited game depth (13 moves per player); and it has limited but not insignificant interaction among players. Several strategies based on locally greedy players, Monte Carlo Evaluation (MCE), and Monte Carlo Tree Search (MCTS) are presented with variants. We examine a variation of UCT called progressive win bias and a playout policy (Player-greedy) focused on selecting good moves for the player. A thorough evaluation is done showing how the strategies pe…
The game of monopoly is an adversarial multi-agent domain where there is no fixed goal other than to be the last player solvent, There are useful subgoals like monopolizing sets of properties, and developing them. There is also a lot of randomness from dice rolls, card-draws, and adversaries' strategies. This unpredictability is made worse when unknown novelties are added during gameplay. Given these challenges, Monopoly was one of the test beds chosen for the DARPA-SAILON program which aims to create agents that can detect and accommodate novelties. To handle the game complexities, we develop…
Researchers on artificial intelligence have achieved human-level intelligence in large-scale perfect-information games, but it is still a challenge to achieve (nearly) optimal results (in other words, an approximate Nash Equilibrium) in large-scale imperfect-information games (i.e. war games, football coach or business strategies). Neural Fictitious Self Play (NFSP) is an effective algorithm for learning approximate Nash equilibrium of imperfect-information games from self-play without prior domain knowledge. However, it relies on Deep Q-Network, which is off-line and is hard to converge in on…
CAST: Game Solvers as Turn-Level Teachers for LLM Agents
研究如何训练大语言模型在长期游戏中决策，提出CAST方法，利用游戏求解器状态值变化转化为求解器优势并注入RLVR，在多个游戏中优于基线，实现高平均零样本性能。
Minesweeper as a puzzle video game and is proved that it is an NPC problem. We use CSP, Logic Inference and Sampling to make a minesweeper solver and we limit us each select in 5 seconds.
Dynamic games are an effective paradigm for dealing with the control of multiple interacting actors. This paper introduces ALGAMES (Augmented Lagrangian GAME-theoretic Solver), a solver that handles trajectory optimization problems with multiple actors and general nonlinear state and input constraints. Its novelty resides in satisfying the first order optimality conditions with a quasi-Newton root-finding algorithm and rigorously enforcing constraints using an augmented Lagrangian formulation. We evaluate our solver in the context of autonomous driving on scenarios with a strong level of inter…
We present the design of a competitive artificial intelligence for Scopone, a popular Italian card game. We compare rule-based players using the most established strategies (one for beginners and two for advanced players) against players using Monte Carlo Tree Search (MCTS) and Information Set Monte Carlo Tree Search (ISMCTS) with different reward functions and simulation strategies. MCTS requires complete information about the game state and thus implements a cheating player while ISMCTS can deal with incomplete information and thus implements a fair player. Our results show that, as expected…
Steering Language Models with Game-Theoretic Solvers
该研究提出一个框架，将博弈论均衡求解器应用于LLM对话空间，通过建模玩家、策略和收益，引导模型生成更稳健、收益更高的谈判语言。
After the recent groundbreaking results of AlphaGo, we have seen a strong interest in reinforcement learning in game playing. General Game Playing (GGP) provides a good testbed for reinforcement learning. In GGP, a specification of games rules is given. GGP problems can be solved by reinforcement learning. Q-learning is one of the canonical reinforcement learning methods, and has been used by (Banerjee & Stone, IJCAI 2007) in GGP. In this paper we implement Q-learning in GGP for three small-board games (Tic-Tac-Toe, Connect Four, Hex), to allow comparison to Banerjee et al. As expected, Q-lear…
Towards a Characterisation of Monte-Carlo Tree Search Performance in Different Games
针对MCTS变体在不同游戏中的性能差异缺乏系统理解的问题，本文构建含1494款游戏、61个智能体、268386次对局的数据集，并训练预测模型以表征其性能。
Real-Time Strategy (RTS) game unit generation is an unexplored area of Procedural Content Generation (PCG) research, which leaves the question of how to automatically generate interesting and balanced units unanswered. Creating unique and balanced units can be a difficult task when designing an RTS game, even for humans. Having an automated method of designing units could help developers speed up the creation process as well as find new ideas. In this work we propose a method of generating balanced and useful RTS units. We draw on Search-Based PCG and a fitness function based on Monte Carlo Tr…
In this paper, we study the effects of several Monte Carlo Tree Search (MCTS) modifications for video game testing. Although MCTS modifications are highly studied in game playing, their impacts on finding bugs are blank. We focused on bug finding in our previous study where we introduced synthetic and human-like test goals and we used these test goals in Sarsa and MCTS agents to find bugs. In this study, we extend the MCTS agent with several modifications for game testing purposes. Furthermore, we present a novel tree reuse strategy. We experiment with these modifications by testing them on th…
We present Solrex,an automated solver for the game of Reverse Hex.Reverse Hex, also known as Rex, or Misere Hex, is the variant of the game of Hex in which the player who joins her two sides loses the game. Solrex performs a mini-max search of the state space using Scalable Parallel Depth First Proof Number Search, enhanced by the pruning of inferior moves and the early detection of certain winning strategies. Solrex is implemented on the same code base as the Hex program Solver, and can solve arbitrary positions on board sizes up to 6x6, with the hardest position taking less than four hours o…
Minesweeper is an interesting single player game based on logic, memory and guessing. Solving Minesweeper has been shown to be an NP-hard task. Deterministic solvers are the best known approach for solving Minesweeper. This project proposes a neural network based learner for solving Minesweeper. To choose the best learner, different architectures and configurations of neural networks were trained on hundreds of thousands of games. Surprisingly, the proposed neural network based learner has shown to be a very good approximation function for solving Minesweeper. The neural network learner compet…
Games are often designed to shape player behavior in a desired way; however, it can be unclear how design decisions affect the space of behaviors in a game. Designers usually explore this space through human playtesting, which can be time-consuming and of limited effectiveness in exhausting the space of possible behaviors. In this paper, we propose the use of automated planning agents to simulate humans of varying skill levels to generate game playthroughs. Metrics can then be gathered from these playthroughs to evaluate the current game design and identify its potential flaws. We demonstrate …
Monte Carlo Tree Search (MCTS) is a relatively new sampling method with multiple variants in the literature. They can be applied to a wide variety of challenging domains including board games, video games, and energy-based problems to mention a few. In this work, we explore the use of the vanilla MCTS and the MCTS with Rapid Action Value Estimation (MCTS-RAVE) in the game of Carcassonne, a stochastic game with a deceptive scoring system where limited research has been conducted. We compare the strengths of the MCTS-based methods with the Star2.5 algorithm, previously reported to yield competit…
Decision-making problems can be modeled as combinatorial optimization problems with Constraint Programming formalisms such as Constrained Optimization Problems. However, few Constraint Programming formalisms can deal with both optimization and uncertainty at the same time, and none of them are convenient to model problems we tackle in this paper. Here, we propose a way to deal with combinatorial optimization problems under uncertainty within the classical Constrained Optimization Problems formalism by injecting the Rank Dependent Utility from decision theory. We also propose a proof of concept…
Advances in intelligent game playing agents have led to successes in perfect information games like Go and imperfect information games like Poker. The Information Set Monte Carlo Tree Search (ISMCTS) family of algorithms outperforms previous algorithms using Monte Carlo methods in imperfect information games. In this paper, Single Observer Information Set Monte Carlo Tree Search (SO-ISMCTS) is applied to Secret Hitler, a popular social deduction board game that combines traditional hidden role mechanics with the randomness of a card deck. This combination leads to a more complex information mo…
Strategy video games challenge AI agents with their combinatorial search space caused by complex game elements. State abstraction is a popular technique that reduces the state space complexity. However, current state abstraction methods for games depend on domain knowledge, making their application to new games expensive. State abstraction methods that require no domain knowledge are studied extensively in the planning domain. However, no evidence shows they scale well with the complexity of strategy games. In this paper, we propose Elastic MCTS, an algorithm that uses state abstraction to pla…
Enhancements for Real-Time Monte-Carlo Tree Search in General Video Game Playing
本文针对通用视频游戏博弈中实时蒙特卡洛树搜索提出八种增强方法，单独应用多数可显著提升胜率，组合后使六十款游戏平均胜率从31.0%提升至48.4%，接近2015年GVG-AI竞赛顶尖水平。
Monte Carlo Tree Search (MCTS) methods have proven powerful in planning for sequential decision-making problems such as Go and video games, but their performance can be poor when the planning depth and sampling trajectories are limited or when the rewards are sparse. We present an adaptation of PGRD (policy-gradient for reward-design) for learning a reward-bonus function to improve UCT (a MCTS algorithm). Unlike previous applications of PGRD in which the space of reward-bonus functions was limited to linear functions of hand-coded state-action-features, we use PGRD with a multi-layer convoluti…
Development and Application of a Monte Carlo Tree Search Algorithm for Simulating Da Vinci Code Game Strategies
本研究开发并评估了蒙特卡洛树搜索算法的两个变体，发现达芬奇密码游戏的分支发散在GPU上导致非线性性能与低谷，而CPU实现呈线性提升，为并行架构下的策略优化提供了关键见解。
Probabilistic inference in the hybrid domain, i.e. inference over discrete-continuous domains, requires tackling two well known #P-hard problems 1)~weighted model counting (WMC) over discrete variables and 2)~integration over continuous variables. For both of these problems inference techniques have been developed separately in order to manage their #P-hardness, such as knowledge compilation for WMC and Monte Carlo (MC) methods for (approximate) integration in the continuous domain. Weighted model integration (WMI), the extension of WMC to the hybrid domain, has been proposed as a formalism to…
Perfect Information Monte Carlo with Postponing Reasoning
针对不完美信息博弈中策略融合问题，提出扩展完美信息蒙特卡洛（EPIMC）算法，通过延迟完美信息解析增强PIMC性能，实证显示其在策略融合影响显著的博弈中表现提升，为相关算法提供理论基础。
In many games, moves consist of several decisions made by the player. These decisions can be viewed as separate moves, which is already a common practice in multi-action games for efficiency reasons. Such division of a player move into a sequence of simpler / lower level moves is called \emph{splitting}. So far, split moves have been applied only in forementioned straightforward cases, and furthermore, there was almost no study revealing its impact on agents' playing strength. Taking the knowledge-free perspective, we aim to answer how to effectively use split moves within Monte-Carlo Tree Sea…
Quantitative Evaluation of Quantum/Classical Neural Network Using a Game Solver Metric
本文提出基于井字棋Elo评分的游戏求解基准，比较经典、量子和混合神经网络，发现混合网络性能接近经典网络，且量子通信开销适度，验证了游戏基准评估量子系统的可行性。
Proof-Number Search (PNS) and Monte-Carlo Tree Search (MCTS) have been successfully applied for decision making in a range of games. This paper proposes a new approach called PN-MCTS that combines these two tree-search methods by incorporating the concept of proof and disproof numbers into the UCT formula of MCTS. Experimental results demonstrate that PN-MCTS outperforms basic MCTS in several games including Lines of Action, MiniShogi, Knightthrough, and Awari, achieving win rates up to 94.0%.
Massively Parallel Proof-Number Search for Impartial Games and Beyond
本文提出了一种大规模并行证明数搜索算法，用于高效求解无偏游戏，并在Sprouts游戏中验证了猜想，实现了显著的加速和复杂度提升。
Beyond Games: A Systematic Review of Neural Monte Carlo Tree Search Applications
本文系统综述了129篇神经蒙特卡洛树搜索在游戏外领域的应用，分析其结构与实践，旨在评估其成功能否扩展至其他领域，并为特定问题设计算法提供原则性指导。
Injecting Combinatorial Optimization into MCTS: Application to the Board Game boop
本文提出将组合优化注入 Monte Carlo Tree Search 以增强博弈树搜索，并在 boop. 中通过消融实验和人类对战验证了其显著性能。
Multi-Agent Training beyond Zero-Sum with Correlated Equilibrium Meta-Solvers
提出JPSRO算法，用于n人一般和扩展型博弈中的智能体训练，证明收敛到均衡，并引入最大基尼关联均衡（MGCE）作为高效的关联均衡元求解器，实验验证其收敛性。
Emerson-Lei and Manna-Pnueli Games for LTLf+ and PPLTL+ Synthesis
针对基于Manna-Pnueli层级的LTLfp和PPLTLp逻辑，提出首个反应式综合求解器：基于Emerson-Lei博弈的符号求解器及原生嵌入目标的Manna-Pnueli博弈求解器，实验证明后者常更高效，结合两者可提升性能。
Turbocharging Solution Concepts: Solving NEs, CEs and CCEs with Neural Equilibrium Solvers
本文提出神经均衡求解器，利用等变神经网络快速确定地近似求解固定形状博弈中的纳什均衡、相关均衡和粗相关均衡，并支持灵活的均衡选择，展现出对更大博弈的零样本泛化能力。
Self-adaptive PSRO: Towards an Automatic Population-based Game Solver
针对PSRO框架依赖手工调参的应用瓶颈，研究提出参数化PSRO及基于Transformer离线超参数优化的自适应PSRO（SPSRO），在多种两人零和博弈上表现优于基线方法。
While Monte Carlo Tree Search and closely related methods have dominated General Video Game Playing, recent research has demonstrated the promise of Rolling Horizon Evolutionary Algorithms as an interesting alternative. However, there is little attention paid to population initialization techniques in the setting of general real-time video games. Therefore, this paper proposes the use of population seeding to improve the performance of Rolling Horizon Evolution and presents the results of two methods, One Step Look Ahead and Monte Carlo Tree Search, tested on 20 games of the General Video Game…
Much current research in AI and games is being devoted to Monte Carlo search (MCS) algorithms. While the quest for a single unified MCS algorithm that would perform well on all problems is of major interest for AI, practitioners often know in advance the problem they want to solve, and spend plenty of time exploiting this knowledge to customize their MCS algorithm in a problem-driven way. We propose an MCS algorithm discovery scheme to perform this in an automatic and reproducible way. We first introduce a grammar over MCS algorithms that enables inducing a rich space of candidate algorithms. …
A Benchmark for Multi-Party Negotiation Games from Real Negotiation Data
本文提出一个多党谈判游戏基准测试，结合可配置的谈判游戏生成器和基于文档的实例，评估不同策略下的谈判方法性能。
Neuro-Symbolic Sudoku Solver
本文扩展神经逻辑机（NLM）架构解决9X9数独问题，通过强化学习训练使模型在3到10个空单元格的数独求解中达到100%准确率，并对比回溯算法验证了符号学习在混合模型中的重要性。
Accelerating Nash Equilibrium Convergence in Monte Carlo Settings Through Counterfactual Value Based Fictitious Play
针对多数CFR改进无法用于蒙特卡洛条件的问题，提出MCCFVFP，融合反事实价值与虚拟博弈最优响应，在高比例被占优策略博弈中收敛更快。
This paper investigates a population-based training regime based on game-theoretic principles called Policy-Spaced Response Oracles (PSRO). PSRO is general in the sense that it (1) encompasses well-known algorithms such as fictitious play and double oracle as special cases, and (2) in principle applies to general-sum, many-player games. Despite this, prior studies of PSRO have been focused on two-player zero-sum games, a regime wherein Nash equilibria are tractably computable. In moving from two-player zero-sum games to more general settings, computation of Nash equilibria quickly becomes infe…
Improve Value Estimation of Q Function and Reshape Reward with Monte Carlo Tree Search
本文针对非完美信息游戏Uno中Q值高估和奖励稀疏问题，提出利用蒙特卡洛树搜索平均Q值估计并重塑奖励的通用算法，实验证明其优于多种传统方法。
Learning strategies for imperfect information games from samples of interaction is a challenging problem. A common method for this setting, Monte Carlo Counterfactual Regret Minimization (MCCFR), can have slow long-term convergence rates due to high variance. In this paper, we introduce a variance reduction technique (VR-MCCFR) that applies to any sampling variant of MCCFR. Using this technique, per-iteration estimated values and updates are reformulated as a function of sampled values and state-action baselines, similar to their use in policy gradient reinforcement learning. The new formulati…
Layered and Staged Monte Carlo Tree Search for SMT Strategy Synthesis
本文提出基于分层分阶段蒙特卡洛树搜索的SMT策略自动合成方法Z3alpha，以低成本高效探索策略空间，在多个基准上超越FastSMT、默认Z3和CVC5。
PokerGPT: An End-to-End Lightweight Solver for Multi-Player Texas Hold'em via Large Language Model
本文提出基于轻量级大语言模型的端到端多人德州扑克求解器 PokerGPT，通过将真实对局文本转换为提示并利用人类反馈强化学习微调，在胜率、模型规模、训练时间和响应速度上优于以往方法。
Domain knowledge is crucial for effective performance in autonomous control systems. Typically, human effort is required to encode this knowledge into a control algorithm. In this paper, we present an approach to language grounding which automatically interprets text in the context of a complex control application, such as a game, and uses domain knowledge extracted from the text to improve control performance. Both text analysis and control strategies are learned jointly using only a feedback signal inherent to the application. To effectively leverage textual information, our method automatic…
Monte Carlo planners can often return sub-optimal actions, even if they are guaranteed to converge in the limit of infinite samples. Known asymptotic regret bounds do not provide any way to measure confidence of a recommended action at the conclusion of search. In this work, we prove bounds on the sub-optimality of Monte Carlo estimates for non-stationary bandits and Markov decision processes. These bounds can be directly computed at the conclusion of the search and do not require knowledge of the true action-value. The presented bound holds for general Monte Carlo solvers meeting mild converg…
The article presents the use of Monte Carlo Tree Search algorithms for the card game Lord of the Rings. The main challenge was the complexity of the game mechanics, in which each round consists of 5 decision stages and 2 random stages. To test various decision-making algorithms, a game simulator has been implemented. The research covered an agent based on expert rules, using flat Monte-Carlo search, as well as complete MCTS-UCB. Moreover different playout strategies has been compared. As a result of experiments, an optimal (assuming a limited time) combination of algorithms were formulated. Th…
UNSAT Solver Synthesis via Monte Carlo Forest Search
本研究提出蒙特卡洛森林搜索（MCFS）类强化学习算法，将其具体化为Knuth Synthesis算法学习SAT问题的DPLL分支策略，通过路径采样估树规模、限定深度调用强求解器降本，在三个SAT分布上性能追平或超越强基线，处理问题难度较此前RL研究高两个数量级。
A Unified Approach to Reinforcement Learning, Quantal Response Equilibria, and Two-Player Zero-Sum Games
本文提出磁性镜像下降算法，统一解决双人零和博弈中的均衡求解与强化学习问题，首次实现一阶反馈下扩展式博弈分位数响应均衡的线性收敛，并在表格设置中达到与CFR相当的性能。
Generative Evolutionary Meta-Solver (GEMS): Scalable Surrogate-Free Multi-Agent Reinforcement Learning
GEMS 提出了一种无代理的多智能体强化学习框架，通过生成器和潜在锚点替代传统种群方法，实现更快的训练速度和更低的内存消耗，同时保持博弈论保证。
How resources are deployed to secure critical targets in networks can be modelled by Network Security Games (NSGs). While recent advances in deep learning (DL) provide a powerful approach to dealing with large-scale NSGs, DL methods such as NSG-NFSP suffer from the problem of data inefficiency. Furthermore, due to centralized control, they cannot scale to scenarios with a large number of resources. In this paper, we propose a novel DL-based method, NSGZero, to learn a non-exploitable policy in NSGs. NSGZero improves data efficiency by performing planning with neural Monte Carlo Tree Search (MC…
MC$^2$: Monte Carlo Correction for Fast Elliptic PDE Solving
MC$^2$结合传统蒙特卡洛方法与神经网络，通过学习单次前向传递的校正来提升PDE求解精度，实现比传统方法快约1000倍的求解速度，并提供首个大规模标准化椭圆型PDE基准PDEZoo。
Which Nash Equilibrium? Solver-Dependent Selection on Zero-Sum Nash Polytopes
研究零和博弈中不同算法如何从纳什均衡多面体中选择特定均衡，发现正则化最后迭代方法选择最大熵成员，而遗憾平均方法漂移到低熵面，且选择影响对次优对手的表现。
扫码添加微信好友，提出您的宝贵建议 👇
💡 备注请填写：网站反馈
发表机构由 AI 从论文文本抽取，如发现错误，欢迎反馈修正建议。