arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2102.08668math.PRcs.LGstat.ML
收藏

高斯过程对神经网络的非渐近逼近

Non-asymptotic approximations of neural networks by Gaussian processes

Weizmann Institute(魏茨曼研究所);Stanford University(斯坦福大学)

Ronen Eldan, Dan Mikulincer, Tselil Schramm

摘要:

我们研究了在何种程度上广泛的神经网络可以近似高斯过程时,初始化随机权重。这是一个公认的事实,当网络的宽度达到无穷大时,它的规律收敛于高斯过程的规律。我们建立明确的收敛速度的中心极限定理在一个无限维的功能空间,度量与自然运输距离的定量。我们确定了两个制度的利益,当激活函数是多项式,其程度决定的收敛速率,而对于非多项式激活,速率是由平滑的功能。

英文摘要:

We study the extent to which wide neural networks may be approximated by Gaussian processes when initialized with random weights. It is a well-established fact that as the width of a network goes to infinity, its law converges to that of a Gaussian process. We make this quantitative by establishing explicit convergence rates for the central limit theorem in an infinite-dimensional functional space, metrized with a natural transportation distance. We identify two regimes of interest; when the activation function is polynomial, its degree determines the rate of convergence, while for non-polynomial activations, the rate is governed by the smoothness of the function.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑