任意目标函数在任意充分宽随机网络邻域内的收敛性:一个几何观点
Any Target Function Exists in a Neighborhood of Any Sufficiently Wide Random Network: A Geometrical Perspective
摘要:
众所周知,任何目标函数都可以在任何随机连接的深度网络的足够小的邻域中实现,前提是宽度(层中神经元的数量)足够大。关于这个惊人的事实,有一些复杂的理论和讨论,但严格的理论是非常复杂的。为了阐明它的结构,我们用一个简单的模型给出了一个初等的几何证明。我们发现,高维几何起着神奇的作用:当我们投影一个半径为1的高维球体到一个低维子空间,在球体上的均匀分布减少到高斯分布的可忽略的小协方差。
英文摘要:
It is known that any target function is realized in a sufficiently small neighborhood of any randomly connected deep network, provided the width (the number of neurons in a layer) is sufficiently large. There are sophisticated theories and discussions concerning this striking fact, but rigorous theories are very complicated. We give an elementary geometrical proof by using a simple model for the purpose of elucidating its structure. We show that high-dimensional geometry plays a magical role: When we project a high-dimensional sphere of radius 1 to a low-dimensional subspace, the uniform distribution over the sphere reduces to a Gaussian distribution of negligibly small covariances.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。