We consider a distributed learning approach in supervised learning for a large class of spectral regularization methods in an RKHS framework. The data set of size n is partitioned into $m=O(n^α)$ disjoint subsets. On each subset, some spectral regularization method (belonging to a large class, including in particular Kernel Ridge Regression, $L^2$-boosting and spectral cut-off) is applied. The regression function $f$ is then estimated via simple averaging, leading to a substantial reduction in computation time. We show that minimax optimal rates of convergence are preserved if m grows sufficiently slowly (corresponding to an upper bound for $α$) as $n \to \infty$, depending on the smoothness assumptions on $f$ and the intrinsic dimensionality. In spirit, our approach is classical.
Spectral Equivariance and Geometric Transport in Reproducing Kernel Hilbert Spaces: A Unified Framework for Orthogonal Polynomial and Kernel Estimation
L.I.A.G.E, Institut National des Sciences de Gestion(国立管理科学研究所);Modeling and Calculus Lab, ROOTS-INSIGHTS(ROOTS-INSIGHTS 建模与计算实验室)
Alignment-Sensitive Minimax Rates for Spectral Algorithms with Learned Kernels
Department of Statistics and Data Science, National University of Singapore, Singapore(新加坡国立大学统计与数据科学系);School of Statistics and Mathematics, Zhongnan University of Economics and Law, Wuhan, China(中南财经政法大学统计与数学学院);Department of Statistics and Data Science, Tsinghua University, Beijing, China(清华大学统计与数据科学系)
估计再生核希尔伯特空间(RKHS)中的核均值的问题是核方法的核心,因为它被经典方法(例如,当将核PCA矩阵居中时),并且它还形成现代核方法的核心推理步骤(例如,基于核的非参数检验),其依赖于在RKHS中嵌入概率分布。Muandet等人(2014)已经表明,收缩可以帮助构建比经验估计量“更好”的核均值估计量。本文研究了Muandet et al.(2014)中估计量的相容性和可容许性,并提出了一类更广泛的收缩估计量,通过考虑适当的基函数来改进经验估计量。使用核PCA的基础上,我们表明,这些估计可以使用频谱滤波算法,这是一致的,在一些技术假设。我们的理论分析还揭示了与基于内核的监督学习框架的基本联系。建议的估计是简单的实现和执行良好的实践。
First-order asymptotic expansions for spectral convergence of compact self-adjoint operators on general spectral subsets, with application to kernel Gram matrices
The Gauss Hypergeometric Covariance Kernel for Modeling Second-Order Stationary Random Fields in Euclidean Spaces: its Compact Support, Properties and Spectral Representation