arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1801.10437cs.AI
收藏

深度学习在实践中发挥作用。但它在理论上有效吗?

Deep Learning Works in Practice. But Does it Work in Theory?

EPFL(洛桑联邦理工学院 (EPFL))

Lê Nguyên Hoang, Rachid Guerraoui

摘要:

深度学习依赖于一种非常特殊的神经网络:那些叠加了几个神经层的网络。近几年来,深度学习在图像分析、语音识别、自然语言处理等领域取得了重大突破,但目前还没有理论解释。特别是,目前尚不清楚为什么网络越深,它实际上表现得越好。我们认为,这种解释与从我们周围的宇宙中收集的数据的一个关键特征密切相关,这些数据是为了满足机器学习算法的需要:大的不可并行的逻辑深度。粗略地说,我们推测,宇宙的最短计算描述是具有固有的大计算时间的算法,即使有大量的计算机可用于并行化。有趣的是,这个猜想,结合理论计算机科学中的民间传说猜想$ P \neq NC$,解释了深度学习的成功。

英文摘要:

Deep learning relies on a very specific kind of neural networks: those superposing several neural layers. In the last few years, deep learning achieved major breakthroughs in many tasks such as image analysis, speech recognition, natural language processing, and so on. Yet, there is no theoretical explanation of this success. In particular, it is not clear why the deeper the network, the better it actually performs. We argue that the explanation is intimately connected to a key feature of the data collected from our surrounding universe to feed the machine learning algorithms: large non-parallelizable logical depth. Roughly speaking, we conjecture that the shortest computational descriptions of the universe are algorithms with inherently large computation times, even when a large number of computers are available for parallelization. Interestingly, this conjecture, combined with the folklore conjecture in theoretical computer science that $ P \neq NC$, explains the success of deep learning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑