arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2102.10307math.PRcs.LGstat.ML
收藏

深度高斯神经网络的大宽度函数渐近性

Large-width functional asymptotics for deep Gaussian neural networks

University of Torino(都灵大学);Collegio Carlo Alberto(卡洛·阿尔贝托学院);Bocconi University(博科尼大学);Cogent Labs(科根特实验室)

Daniele Bracale, Stefano Favaro, Sandra Fortini, Stefano Peluchetti

AI总结:

本文针对权重和偏置服从独立同分布高斯分布的全连接前馈深度神经网络,采用函数空间视角,证明其定义连续高斯过程、权重重缩放网络大宽度极限下弱收敛于连续高斯过程且极限过程几乎必然有局部$\gamma$-Hölder连续路径,为相关理论研究贡献。

摘要:

本文考虑全连接前馈深度神经网络,其权重和偏置服从独立同分布的高斯分布。扩展先前结果(Matthews等人,2018a;b;Yang,2019),我们采用函数空间视角,即将神经网络视为输入空间$\mathbb{R}^I$上的无限维随机元素。在激活函数的适当假设下,我们证明:i)网络在输入空间$\mathbb{R}^I$上定义了一个连续高斯过程;ii)权重重新缩放的网络在大宽度极限下弱收敛到连续高斯过程;iii)极限高斯过程几乎必然具有局部$\gamma$-Hölder连续路径($0 < \gamma<1$)。我们的结果通过建立函数空间中关于更强度量的弱收敛,为无限宽深度神经网络与高斯过程之间相互作用的近期理论研究做出贡献。

英文摘要:

In this paper, we consider fully connected feed-forward deep neural networks where weights and biases are independent and identically distributed according to Gaussian distributions. Extending previous results (Matthews et al., 2018a;b; Yang, 2019) we adopt a function-space perspective, i.e. we look at neural networks as infinite-dimensional random elements on the input space $\mathbb{R}^I$. Under suitable assumptions on the activation function we show that: i) a network defines a continuous Gaussian process on the input space $\mathbb{R}^I$; ii) a network with re-scaled weights converges weakly to a continuous Gaussian process in the large-width limit; iii) the limiting Gaussian process has almost surely locally $γ$-Hölder continuous paths, for $0 < γ<1$. Our results contribute to recent theoretical studies on the interplay between infinitely wide deep neural networks and Gaussian processes by establishing weak convergence in function-space with respect to a stronger metric.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑