统一的单玩家和多玩家表演预测推断框架：方法与渐近最优性

面向离线零和 Markov 博弈的基于模型的强化学习

Model-Based Reinforcement Learning for Offline Zero-Sum Markov Games

针对离线两人零和 Markov 博弈的 Nash 均衡学习，提出悲观基于模型算法 VI-LCB-Game，在全准确率范围内取得改进 min{A,B} 倍的 minimax 最优样本复杂度。

针对适当约束假设的最优 e-value 检验

Optimal e-value testing for properly constrained hypotheses

本文研究了基于 e-variables 的序贯假设检验，通过限制 e-variable 池来简化策略设计，扩展了 Clerico(2024) 的结果，刻画了有限正则约束下非参数检验的最优 e-variable 集合，并应用于算法均值估计。

Nearly Minimax Optimal Offline Reinforcement Learning with Linear Function Approximation: Single-Agent MDP and Markov Game

Offline reinforcement learning (RL) aims at learning an optimal strategy using a pre-collected dataset without further interactions with the environment. While various algorithms have been proposed for offline RL in the previous literature, the minimax optimality has only been (nearly) established for tabular Markov decision processes (MDPs). In this paper, we focus on offline RL with linear function approximation and propose a new pessimism-based algorithm for offline linear MDP. At the core of our algorithm is the uncertainty decomposition via a reference function, which is new in the litera…

What is Local Optimality in Nonconvex-Nonconcave Minimax Optimization?

Minimax optimization has found extensive applications in modern machine learning, in settings such as generative adversarial networks (GANs), adversarial training and multi-agent reinforcement learning. As most of these applications involve continuous nonconvex-nonconcave formulations, a very basic question arises---"what is a proper definition of local optima?" Most previous work answers this question using classical notions of equilibria from simultaneous games, where the min-player and the max-player act simultaneously. In contrast, most applications in machine learning, including GANs and …

Almost Optimal Algorithms for Two-player Zero-Sum Linear Mixture Markov Games

We study reinforcement learning for two-player zero-sum Markov games with simultaneous moves in the finite-horizon setting, where the transition kernel of the underlying Markov games can be parameterized by a linear function over the current state, both players' actions and the next state. In particular, we assume that we can control both players and aim to find the Nash Equilibrium by minimizing the duality gap. We propose an algorithm Nash-UCRL based on the principle "Optimism-in-Face-of-Uncertainty". Our algorithm only needs to find a Coarse Correlated Equilibrium (CCE), which is computatio…

Multi-Player Bandits Revisited

Multi-player Multi-Armed Bandits (MAB) have been extensively studied in the literature, motivated by applications to Cognitive Radio systems. Driven by such applications as well, we motivate the introduction of several levels of feedback for multi-player MAB algorithms. Most existing work assume that sensing information is available to the algorithm. Under this assumption, we improve the state-of-the-art lower bound for the regret of any decentralized algorithms and introduce two algorithms, RandTopM and MCTopM, that are shown to empirically outperform existing algorithms. Moreover, we provide…

两人人零和博弈中乐观 Hedge 算法的的紧致与宽松的界和下界的上下界与下界

Tight Regret Upper and Lower Bounds for Optimistic Hedge in Two-Player Zero-Sum Games

本文研究两人零和博弈中乐观Hedge算法遗憾界对动作数量m和n的依赖最优性。通过将遗憾上界转化为优化问题，在已知对手动作数时将社会与个体遗憾界改进至O(sqrt(log m log n))，并给出匹配的算法依赖下界，最后提升了算法的末次迭代收敛率与动态遗憾界。

面向多层级预算组合问题的课程学习

Curriculum learning for multilevel budgeted combinatorial problems

针对图上两人零和的多层级预算组合优化问题，提出基于价值的多智能体强化学习与预算递增课程训练方法，在百节点图上接近最优，并较最快精确求解器平均加速185倍。

A Goal-Based Movement Model for Continuous Multi-Agent Tasks

Despite increasing attention paid to the need for fast, scalable methods to analyze next-generation neuroscience data, comparatively little attention has been paid to the development of similar methods for behavioral analysis. Just as the volume and complexity of brain data have grown, behavioral paradigms in systems neuroscience have likewise become more naturalistic and less constrained, necessitating an increase in the flexibility and scalability of the models used to study them. In particular, key assumptions made in the analysis of typical decision paradigms --- optimality; analytic tract…

Representation Learning for General-sum Low-rank Markov Games

We study multi-agent general-sum Markov games with nonlinear function approximation. We focus on low-rank Markov games whose transition matrix admits a hidden low-rank structure on top of an unknown non-linear representation. The goal is to design an algorithm that (1) finds an $\varepsilon$-equilibrium policy sample efficiently without prior knowledge of the environment or the representation, and (2) permits a deep-learning friendly implementation. We leverage representation learning and present a model-based and a model-free approach to construct an effective representation from the collecte…