arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1602.00355stat.MEstat.ML
收藏

高维非参数回归的谱序列方法

A Spectral Series Approach to High-Dimensional Nonparametric Regression

Ann B. Lee, Rafael Izbicki

摘要:

现代统计学中的一个关键问题是如何对复杂的高维数据进行快速可靠的推断。虽然稀疏技术引起了人们的极大兴趣,但目前的方法不能很好地推广到具有非线性结构的数据。在这项工作中,我们提出了一个正交系列的预测是复杂的聚合对象,如自然图像,星系光谱,轨迹和电影估计。我们的系列方法将内核机器学习和傅立叶方法的思想联系在一起。我们扩展未知回归的数据方面的特征函数的内核为基础的运营商,我们利用正交的基础上的基本数据分布,P,以加快计算和调整参数。如果核被适当地选择,则特征函数适应于数据的内在几何形状和维度。我们提供了理论上的保证,径向内核具有不同的带宽,我们与光滑的回归函数相对于P稀疏的特征基。最后,使用模拟和真实世界的数据,我们系统地比较了经典的核平滑,k-最近邻回归,核岭回归,和国家的最先进的流形和局部回归方法的性能的谱序列方法。

英文摘要:

A key question in modern statistics is how to make fast and reliable inferences for complex, high-dimensional data. While there has been much interest in sparse techniques, current methods do not generalize well to data with nonlinear structure. In this work, we present an orthogonal series estimator for predictors that are complex aggregate objects, such as natural images, galaxy spectra, trajectories, and movies. Our series approach ties together ideas from kernel machine learning, and Fourier methods. We expand the unknown regression on the data in terms of the eigenfunctions of a kernel-based operator, and we take advantage of orthogonality of the basis with respect to the underlying data distribution, P, to speed up computations and tuning of parameters. If the kernel is appropriately chosen, then the eigenfunctions adapt to the intrinsic geometry and dimension of the data. We provide theoretical guarantees for a radial kernel with varying bandwidth, and we relate smoothness of the regression function with respect to P to sparsity in the eigenbasis. Finally, using simulated and real-world data, we systematically compare the performance of the spectral series approach with classical kernel smoothing, k-nearest neighbors regression, kernel ridge regression, and state-of-the-art manifold and local regression methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑