How many bits of information are required to PAC learn a class of hypotheses of VC dimension $d$? The mathematical setting we follow is that of Bassily et al. (2018), where the value of interest is the mutual information $\mathrm{I}(S;A(S))$ between the input sample $S$ and the hypothesis outputted by the learning algorithm $A$. We introduce a class of functions of VC dimension $d$ over the domain $\mathcal{X}$ with information complexity at least $Ω\left(d\log \log \frac{|\mathcal{X}|}{d}\right)$ bits for any consistent and proper algorithm (deterministic or random). Bassily et al. proved a similar (but quantitatively weaker) result for the case $d=1$.
The above result is in fact a special case of a more general phenomenon we explore. We define the notion of information complexity of a given class of functions $\mathcal{H}$. Intuitively, it is the minimum amount of information that an algorithm for $\mathcal{H}$ must retain about its input to ensure consistency and properness. We prove a direct sum result for information complexity in this context; roughly speaking, the information complexity sums when combining several classes.
Freya PAGE: First Optimal Time Complexity for Large-Scale Nonconvex Finite-Sum Optimization with Heterogeneous Asynchronous Computations
King Abdullah University of Science and Technology(阿卜杜拉国王科技大学);AIRI;Skolkovo Institute of Science and Technology(斯科尔科沃科学技术研究院);KAUST(阿卜杜拉国王科技大学);Skoltech(斯科尔泰克科技大学)
On the Complexity of Computing Markov Perfect Equilibrium in General-Sum Stochastic Games
Center on Frontiers of Computing Studies(计算研究前沿中心);Peking University(北京大学);Huawei R&D UK(华为英国研发中心);University College London(伦敦大学学院);Institute for AI, Peking University(北京大学人工智能研究院)
类似于马尔可夫决策过程在强化学习中的作用,随机游戏(SG)为多智能体强化学习(MARL)和顺序智能体交互的研究奠定了基础。本文证明了在指数精度下计算有限状态折扣随机博弈的近似马尔可夫完美均衡是\textbf{PPAD} -完全的。我们采用了一个函数与多项式有界的描述在战略空间转换MPE计算到一个不动点问题,即使随机游戏可能需要一个指数数量的纯策略,在国家的数量,为每个代理。完整性结果遵循将定点问题简化为{ \sc End of the Line}。我们的研究结果表明,在SG中发现MPE极不可能是\textbf{NP} -困难的,除非\textbf{NP} = \textbf{co-NP}。我们的工作为MARL研究提供了信心,研究MPE计算一般和SG和开发富有成效的算法,目前零和SG。