arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1702.07800cs.LGcs.NEstat.ML
收藏

关于深度学习的起源

On the Origin of Deep Learning

Carnegie Mellon University(卡内基梅隆大学)

Haohan Wang, Bhiksha Raj

摘要:

本文回顾了深度学习模型的发展历史。它涵盖了从神经网络的起源,当研究大脑的联想主义建模时,到过去十年在深度学习研究中占主导地位的模型,如卷积神经网络,深度信念网络和递归神经网络。除了对这些模型进行回顾之外,本文还主要关注上述模型的先例,研究最初的想法是如何组合起来构建早期模型的,以及这些初步模型是如何发展到目前的形式的。这些进化路径中的许多持续了半个多世纪,并且具有多样性的方向。例如,CNN是建立在生物视觉系统的先验知识上的; DBN是从图形模型的建模能力和计算复杂性的权衡中演变而来的,并且现在许多模型是古代线性模型的神经对应物。本文回顾了这些进化路径,并提供了一个简明的思想流程,这些模型是如何开发的,旨在为深度学习提供一个全面的背景。更重要的是,本文沿着路径,总结了这些里程碑背后的要旨,并提出了许多指导深度学习未来研究的方向。

英文摘要:

This paper is a review of the evolutionary history of deep learning models. It covers from the genesis of neural networks when associationism modeling of the brain is studied, to the models that dominate the last decade of research in deep learning like convolutional neural networks, deep belief networks, and recurrent neural networks. In addition to a review of these models, this paper primarily focuses on the precedents of the models above, examining how the initial ideas are assembled to construct the early models and how these preliminary models are developed into their current forms. Many of these evolutionary paths last more than half a century and have a diversity of directions. For example, CNN is built on prior knowledge of biological vision system; DBN is evolved from a trade-off of modeling power and computation complexity of graphical models and many nowadays models are neural counterparts of ancient linear models. This paper reviews these evolutionary paths and offers a concise thought flow of how these models are developed, and aims to provide a thorough background for deep learning. More importantly, along with the path, this paper summarizes the gist behind these milestones and proposes many directions to guide the future research of deep learning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑