Kernel methods have recently attracted resurgent interest, showing performance competitive with deep neural networks in tasks such as speech recognition. The random Fourier features map is a technique commonly used to scale up kernel machines, but employing the randomized feature map means that $O(ε^{-2})$ samples are required to achieve an approximation error of at most $ε$. We investigate some alternative schemes for constructing feature maps that are deterministic, rather than random, by approximating the kernel in the frequency domain using Gaussian quadrature. We show that deterministic feature maps can be constructed, for any $γ> 0$, to achieve error $ε$ with $O(e^{e^γ} + ε^{-1/γ})$ samples as $ε$ goes to 0. Our method works particularly well with sparse ANOVA kernels, which are inspired by the convolutional layer of CNNs. We validate our methods on datasets in different domains, such as MNIST and TIMIT, showing that deterministic features are faster to generate and achieve accuracy comparable to the state-of-the-art kernel methods based on random Fourier features.
Fast Approximation and Estimation Bounds of Kernel Quadrature for Infinitely Wide Models
RIKEN AIP（日本理化学研究所革新智能综合研究中心）
无限宽模型是特征图的加权积分$\int φ(x,v) d μ(v)$。该模型擅长处理无限数量的特征，因此已被用于深度学习的理论研究。核求积是一种基于核的数值积分方案，用于快速逼近期望值$\int f(x) d p(x)$。在这项研究中，关于重量$μ$作为一个有符号（或复杂/向量值）分布的参数，我们开发的一般核求积（GKQ）的参数分布。所提出的方法可以实现一个快速的近似速度$O(e^{-p})$与参数数$p$，这是比传统的巴伦的速度，和一个快速的估计速度$\widetilde{O}(1/n)$与样本量$n$。因此，我们得到了一个新的基于范数的复杂性度量的无限宽模型。由于GKQ隐式地进行经验风险最小化，因此我们可以理解复杂性度量也反映了梯度学习设置中的泛化性能。
Fixing the Pitfalls of Probabilistic Time-Series Forecasting Evaluation by Kernel Quadrature
Lattice Lab, Toyota Motor Corporation / Machine Learning Research Group, University of Oxford（丰田公司 lattice 实验室 / 机器学习研究组，牛津大学）；The University of Osaka / Lattice Lab, Toyota Motor Corporation / RIKEN AIP（大阪大学 / 丰田公司 lattice 实验室 / RIKEN AIP）；Machine Learning Research Group, University of Oxford（机器学习研究组，牛津大学）
University of Warwick, Department of Statistics（沃里克大学统计系）；Imperial College London, Department of Mathematics（伦敦帝国理工学院数学系）；Newcastle University, School of Mathematics（新castle大学数学系）；University of Technology Sydney, School of Mathematical（悉尼技术大学数学与物理科学学院）；The Alan Turing Institute for Data Science（数据科学阿尔图里研究所）
Gauss--Hermite Quadrature for Gaussian-Mixture Entropy with an Action-Space Hermite Surrogate
Extreme Materials Research Center, Korea Institute of Science and Technology（韩国科学技术研究院极端材料研究中心）；Climate and Environmental Research Institute, Korea Institute of Science and Technology（韩国科学技术研究院气候与环境研究所）；Division of AI-Robotics, KIST Campus, University of Science and Technology（科学技术大学KIST校区AI机器人系）
