arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2012.10931cs.LGstat.ML
收藏

深度学习理论的最新进展

Recent advances in deep learning theory

the University of Sydney(悉尼大学);UBTECH Sydney AI Centre(优必选悉尼人工智能中心)

Fengxiang He, Dacheng Tao

摘要:

深度学习通常被描述为一个实验驱动的领域,不断受到缺乏理论基础的批评。这一问题已部分解决了大量的文献,迄今尚未妥善组织。本文回顾和整理了深度学习理论的最新进展。文献分为六组:(1)用于分析深度学习可推广性的基于复杂性和容量的方法;(2)用于建模随机梯度下降及其变体的随机微分方程及其动态系统,其特征在于深度学习的优化和推广,部分受到贝叶斯推理的启发;(3)驱动动态系统轨迹的损失景观的几何结构;(4)从正反两个角度讨论了深度神经网络的过参数化作用;(5)网络结构中几种特殊结构的理论基础;(6)伦理与安全的日益关注及其与普遍性的关系。

英文摘要:

Deep learning is usually described as an experiment-driven field under continuous criticizes of lacking theoretical foundations. This problem has been partially fixed by a large volume of literature which has so far not been well organized. This paper reviews and organizes the recent advances in deep learning theory. The literature is categorized in six groups: (1) complexity and capacity-based approaches for analyzing the generalizability of deep learning; (2) stochastic differential equations and their dynamic systems for modelling stochastic gradient descent and its variants, which characterize the optimization and generalization of deep learning, partially inspired by Bayesian inference; (3) the geometrical structures of the loss landscape that drives the trajectories of the dynamic systems; (4) the roles of over-parameterization of deep neural networks from both positive and negative perspectives; (5) theoretical foundations of several special structures in network architectures; and (6) the increasingly intensive concerns in ethics and security and their relationships with generalizability.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑