arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2006.03177cs.LGcs.CCstat.ML
收藏

具有自然权值的神经网络学习的困难性

Hardness of Learning Neural Networks with Natural Weights

The Hebrew University(希伯来大学);Google Research(谷歌研究院);Weizmann Institute of Science(魏茨曼科学研究所)

Amit Daniely, Gal Vardi

摘要:

神经网络如今非常成功,尽管结果很难。现有的硬度结果集中在网络结构,并假设网络的权重是任意的。解决这种差异的一种自然方法是假设网络的权重是“行为良好”的,并且具有一些通用属性,可以允许有效的学习。这种方法得到了直觉的支持,即现实世界网络中的权重不是任意的,而是相对于一些“自然”分布表现出一些“类似随机”的特性。我们证明了这方面的负面结果,并表明,对于深度- $2$网络,以及许多“自然”的权重分布,如正态分布和均匀分布,大多数网络是很难学习的。也就是说,没有有效的学习算法可证明对大多数权重和每个输入分布都是成功的。这意味着在这种随机网络中不存在具有高概率的通用属性,并允许有效的学习。

英文摘要:

Neural networks are nowadays highly successful despite strong hardness results. The existing hardness results focus on the network architecture, and assume that the network's weights are arbitrary. A natural approach to settle the discrepancy is to assume that the network's weights are "well-behaved" and posses some generic properties that may allow efficient learning. This approach is supported by the intuition that the weights in real-world networks are not arbitrary, but exhibit some "random-like" properties with respect to some "natural" distributions. We prove negative results in this regard, and show that for depth-$2$ networks, and many "natural" weights distributions such as the normal and the uniform distribution, most networks are hard to learn. Namely, there is no efficient learning algorithm that is provably successful for most weights, and every input distribution. It implies that there is no generic property that holds with high probability in such random networks and allows efficient learning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑