Let $K: \boldsymbolΩ\times \boldsymbolΩ$ be a continuous Mercer kernel defined on a compact subset of ${\mathbb R}^n$ and $\mathcal{H}_K$ be the reproducing kernel Hilbert space (RKHS) associated with $K$. Given a finite measure $ν$ on $\boldsymbolΩ$, we investigate upper and lower bounds on the $\varepsilon$-entropy of the unit ball of $\mathcal{H}_K$ in the space $L_p(ν)$. This topic is an important direction in the modern statistical theory of kernel-based methods.
We prove sharp upper and lower bounds for $p\in [1,+\infty]$. For $p\in [1,2]$, the upper bounds are determined solely by the eigenvalue behaviour of the corresponding integral operator $ϕ\to \int_{\boldsymbolΩ} K(\cdot,{\mathbf y})ϕ({\mathbf y})dν({\mathbf y})$. In constrast, for $p>2$, the bounds additionally depend on the convergence rate of the truncated Mercer series to the kernel $K$ in the $L_p(ν)$-norm.
We discuss a number of consequences of our bounds and show that they are substantially tighter than previous bounds for general kernels. Furthermore, for specific cases, such as zonal kernels and the Gaussian kernel on a box, our bounds are asymptotically tight as $\varepsilon\to +0$.
Learning to Embed Distributions via Maximum Kernel Entropy
Dipartimento di Matematica, Università degli Studi di Genova(热那亚大学数学系);Technion Israel Institute of Technology(以色列理工学院);Allen Institute for Neural Dynamics(艾伦神经动力学研究所)
HSE University(高等经济大学);Artificial Intelligence Research Institute(人工智能研究院);Duisburg-Essen University(杜伊斯堡-埃森大学);Google DeepMind(谷歌DeepMind);École Polytechnique(巴黎综合理工学院);Mohamed Bin Zayed University of AI(穆罕默德·本·扎耶德人工智能大学);IDEMIA(艾德米亚);ENS Lyon(里昂高等师范学院)
KL-Entropy-Regularized RL with a Generative Model is Minimax Optimal
University of Alberta(阿尔伯塔大学);Peking University(北京大学);Google Research(谷歌研究院);Université de Lorraine(洛林大学);CNRS(法国国家科学研究中心);INRIA(法国国家信息与自动化研究所);IECL(洛林计算机科学及其应用实验室);University of Tokyo(东京大学);DeepMind(深度思维);Otto von Guericke University Magdeburg(奥托·冯·格里克马格德堡大学)