深度ReLU神经网络学习谱Barron函数的极小极大速率

中文摘要 AI 辅助

我们研究深度神经网络逼近和学习谱Barron函数的性能。近期研究表明，这些函数类可以通过浅层神经网络高效逼近，而不会遭受维数灾难。我们通过为具有ReLU激活函数的深度网络提供新的逼近界，并建立学习这些函数类的极小极大速率，补充了这些结果。具体而言，我们证明光滑指数为$s>0$的$d$维谱Barron函数可以通过深度ReLU神经网络以逼近速率$\widetilde{\mathcal{O}} (S^{-\frac{1}{2}-\frac{s}{d}})$逼近，其中$S$表示网络中非零参数的个数。利用这一逼近结果，我们进一步证明深度ReLU神经网络可以用$n$个训练样本以快速速率$n^{-\frac{d+2s}{2d+2s}}$学习谱Barron函数。最后，我们证明该收敛速率在忽略对数因子的意义下是极小极大最优的。

英文摘要

We study how well deep neural networks approximate and learn spectral Barron functions. Recent studies have shown that these function classes can be efficiently approximated by shallow neural networks without suffering from the curse of dimensionality. We complement these results by providing new approximation bounds for deep networks with ReLU activation and establishing the minimax rates for learning these function classes. Specifically, we show that $d$-dimensional spectral Barron functions with smoothness index $s>0$ can be approximated by deep ReLU neural networks with approximation rate $\widetilde{\mathcal{O}} (S^{-\frac{1}{2}-\frac{s}{d}})$, where $S$ denotes the number of nonzero parameters in the network. Using this approximation result, we further show that deep ReLU neural networks can learn spectral Barron functions in a fast rate $n^{-\frac{d+2s}{2d+2s}}$ with $n$ training samples. Finally, we prove that this convergence rate is minimax optimal up to logarithmic factors.

发表机构

• Sun Yat-sen University（中山大学）

机构由 AI 辅助整理，请以论文原文为准。