arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2104.06708math.STstat.TH
收藏

近似流形上的深度非参数回归:多项式前因子的非渐近误差界

Deep Nonparametric Regression on Approximate Manifolds: Non-Asymptotic Error Bounds with Polynomial Prefactors

Yuling Jiao, Guohao Shen, Yuanyuan Lin, Jian Huang

摘要:

我们使用深度神经网络研究非参数最小二乘回归的性质。我们推导出前馈深度神经回归的经验风险最小化器的预测误差的非渐近上界。我们的误差界实现最小最大的最佳速率和显着改善现有的意义上,他们依赖于多项式的维度上的预测,而不是指数的维度。我们表明,神经回归估计可以规避灾难的维数的假设下,预测支持一个近似的低维流形或一组具有低Minkowski维。我们还建立了最佳的收敛速度下的确切流形支持假设。本文研究了神经网络结构对神经回归估计器预测误差的影响,提出了两类神经网络之间的网络相对效率的概念,为评价不同网络结构的优劣提供了一种定量的度量。为了建立这些结果,我们使用ReLU激活的神经网络导出了具有正光滑指数的Hölder光滑函数的新近似误差界,这可能是独立的兴趣。我们的结果是在比现有文献中的数据分布和神经网络结构更弱的假设下得出的。

英文摘要:

We study the properties of nonparametric least squares regression using deep neural networks. We derive non-asymptotic upper bounds for the prediction error of the empirical risk minimizer of feedforward deep neural regression. Our error bounds achieve minimax optimal rate and significantly improve over the existing ones in the sense that they depend polynomially on the dimension of the predictor, instead of exponentially on dimension. We show that the neural regression estimator can circumvent the curse of dimensionality under the assumption that the predictor is supported on an approximate low-dimensional manifold or a set with low Minkowski dimension. We also establish the optimal convergence rate under the exact manifold support assumption. We investigate how the prediction error of the neural regression estimator depends on the structure of neural networks and propose a notion of network relative efficiency between two types of neural networks, which provides a quantitative measure for evaluating the relative merits of different network structures. To establish these results, we derive a novel approximation error bound for the Hölder smooth functions with a positive smoothness index using ReLU activated neural networks, which may be of independent interest. Our results are derived under weaker assumptions on the data distribution and the neural network structure than those in the existing literature.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑