通过KL正则化实现一般和博弈中的无悲观离线学习
Pessimism-Free Offline Learning in General-Sum Games via KL Regularization
AI总结：
本文提出了一种基于KL正则化的离线学习方法，能够在一般和博弈中实现无悲观的均衡恢复，通过加速的统计速率和计算高效的算法提升学习效率。
摘要：
在一般和博弈的离线多智能体强化学习中， Logged数据集与目标均衡策略之间的分布偏移是一个挑战。尽管传统方法依赖于手动悲观惩罚，我们证明KL正则化足以稳定学习并实现均衡恢复。我们提出了General-sum Anchored Nash Equilibrium (GANE)，其在加速的统计速率$\widetilde{O}(1/n)$下恢复正则化的纳什均衡。为了计算可行性，我们开发了General-sum Anchored Mirror Descent (GAMD)，一种迭代算法，能够在标准速率$\widetilde{O}(1/\sqrt{n}+1/T)$下收敛到粗相关均衡。这些结果确立了KL正则化作为无悲观离线学习的独立机制，在多玩家一般和博弈中实现等效或加速的速率。
英文摘要：
Offline multi-agent reinforcement learning in general-sum settings is challenged by the distribution shift between logged datasets and target equilibrium policies. While standard methods rely on manual pessimistic penalties, we demonstrate that KL regularization suffices to stabilize learning and achieve equilibrium recovery. We propose General-sum Anchored Nash Equilibrium (GANE), which recovers regularized Nash equilibria at an accelerated statistical rate of $\widetilde{O}(1/n)$. For computational tractability, we develop General-sum Anchored Mirror Descent (GAMD), an iterative algorithm converging to a Coarse Correlated Equilibrium at the standard rate of $\widetilde{O}(1/\sqrt{n}+1/T)$. These results establish KL regularization as a standalone mechanism for pessimism-free offline learning that achieves equivalent or accelerated rates in multi-player general-sum games.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
