arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1111.2262cs.LGcs.NAmath.NA
收藏

Nyström方法的改进界及其在核分类中的应用

Improved Bound for the Nystrom's Method and its Application to Kernel Classification

Michigan State University(密歇根州立大学);Nanjing University(南京大学)

Rong Jin, Tianbao Yang, Mehrdad Mahdavi, Yu-Feng Li, Zhi-Hua Zhou

AI总结:

本文通过积分算子集中不等式和压缩感知理论改进了Nyström方法的谱范数逼近误差界,并应用于核分类,证明在特征值服从p次幂律时可将支持向量数量减少至N^{2p/(p^2-1)}。

摘要:

我们开发了两种分析Nyström方法逼近误差界的方法,一种基于积分算子的集中不等式,另一种基于压缩感知理论。我们表明,在大特征间隙的情况下,以谱范数度量的逼近误差可以从$O(N/\sqrt{m})$改进到$O(N/m^{1 - ρ})$,其中$N$是数据点总数,$m$是采样数据点数,$ρ\in (0, 1/2)$是刻画特征间隙的正常数。当核矩阵的特征值服从$p$次幂律时,基于压缩感知理论的分析在非相干性假设下进一步将界改进为$O(N/m^{p - 1})$,这解释了为什么Nyström方法对特征值倾斜的核矩阵效果良好。我们提出了一种基于Nyström方法的核分类方法,并利用改进的界推导了其泛化性能。我们表明,当核矩阵的特征值服从$p$次幂律时,我们可以将支持向量数量减少到$N^{2p/(p^2 - 1)}$,当$p > 1+\sqrt{2}$时该数量小于$N$,而不会严重牺牲其泛化性能。

英文摘要:

We develop two approaches for analyzing the approximation error bound for the Nyström method, one based on the concentration inequality of integral operator, and one based on the compressive sensing theory. We show that the approximation error, measured in the spectral norm, can be improved from $O(N/\sqrt{m})$ to $O(N/m^{1 - ρ})$ in the case of large eigengap, where $N$ is the total number of data points, $m$ is the number of sampled data points, and $ρ\in (0, 1/2)$ is a positive constant that characterizes the eigengap. When the eigenvalues of the kernel matrix follow a $p$-power law, our analysis based on compressive sensing theory further improves the bound to $O(N/m^{p - 1})$ under an incoherence assumption, which explains why the Nyström method works well for kernel matrix with skewed eigenvalues. We present a kernel classification approach based on the Nyström method and derive its generalization performance using the improved bound. We show that when the eigenvalues of kernel matrix follow a $p$-power law, we can reduce the number of support vectors to $N^{2p/(p^2 - 1)}$, a number less than $N$ when $p > 1+\sqrt{2}$, without seriously sacrificing its generalization performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑