用于蒙特卡洛反事实遗憾最小化的相关机会采样
Correlated Chance Sampling for Monte Carlo Counterfactual Regret Minimization
浏览论文内容
中文总结 AI 辅助
该研究提出相关机会采样MCCFR（CCS-MCCFR），通过修改机会采样器实现局部频率误差更小的采样，在多种扑克游戏中显著降低可利用性，且无额外超参数与时间开销。
中文摘要 AI 辅助
蒙特卡洛反事实遗憾最小化（MCCFR）在策略演化过程中会反复分配机会结果，但标准采样在每次访问时独立抽取这些结果。我们提出了相关机会采样MCCFR（CCS-MCCFR），这是一种可直接替换的方法，它为每个具体的机会节点分配一个持久的随机Weyl流，并通过该节点的机会分布映射其阶段。每个固定索引的抽取具有正确的边际律，而在对一个具体节点的N次访问中消耗的前N次抽取，实现了确定性的局部频率误差O(log(N+1)/N)，相比之下，独立同分布（i.i.d.）频率的预期尺度为O(N^{-1/2})。我们进一步证明了其在固定策略轨迹上的无偏性，通过条件标量界分离了自适应阶段选择，并表明每次遍历重置的变体仍保持标准的O(1/√T)外部采样保证。配对实验中，CCS-MCCFR在库恩扑克和四种Leduc扑克配置下将最终可利用性降低了19.05%至34.01%，所有配对自助法置信区间均大于零，在Goofspiel-4上也显著降低了4.27%。该增益在300万次Leduc节点访问后仍然存在，且与线性CFR结合后达到了已测的最低可利用性。该采样器未引入新的超参数，也无明显时间开销，因此CCS-MCCFR只需对机会采样器做一行修改，即可获得明确的局部保证并在表格型扑克中大幅降低可利用性。
英文摘要
Monte Carlo Counterfactual Regret Minimization (MCCFR) repeatedly allocates chance outcomes while its strategy evolves, yet standard sampling draws those outcomes independently on every visit. We introduce Correlated Chance Sampling MCCFR (CCS-MCCFR), a drop-in replacement that assigns each concrete chance node a persistent randomized Weyl stream and maps its phases through the node's chance distribution. Each fixed-index draw has the correct marginal law, while the first $N$ draws consumed during $N$ visits to one concrete node achieve deterministic local frequency error $O(\!\log(N+1)/N)$, compared with the $O(N^{-1/2})$ expected scale of i.i.d. frequencies. We further establish unbiasedness along fixed strategy trajectories, isolate adaptive phase selection through a conditional scalar bound, and show that a per-traversal reset variant retains the standard $O(1/\sqrt{T})$ External Sampling guarantee. In paired experiments, CCS-MCCFR reduces final exploitability by 19.05\% to 34.01\% across Kuhn poker and four Leduc poker configurations, with every paired-bootstrap confidence interval above zero, and by a significant 4.27\% on Goofspiel-4. The gain survives to 3M Leduc node touches and combines with Linear CFR to reach the lowest measured exploitability. The sampler introduces no new hyperparameters and no measurable time overhead, so CCS-MCCFR turns a one-line change to the chance sampler into explicit local guarantees and large exploitability reductions across tabular poker.
