arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1807.01083math.OCcs.LG
收藏

深度学习的平均场最优控制公式

A Mean-Field Optimal Control Formulation of Deep Learning

Princeton University(普林斯顿大学);Beijing Institute of Big Data Research(北京大数据研究院);Peking University(北京大学);Institute of High Performance Computing, Agency for Science, Technology and Research(科技研究局高性能计算研究院)

Weinan E, Jiequn Han, Qianxiao Li

摘要:

最近将深度神经网络和动力系统联系起来的工作为分析深度学习开辟了新的途径。特别是,可以观察到,通过将深度学习重新塑造为差分或微分方程的最优控制问题,可以获得新的见解。然而,这种提法的数学方面还没有得到系统的探讨。本文介绍了深度学习中群体风险最小化问题的数学表述,将其作为平均场最优控制问题。仿照经典最优控制的发展,我们提出并证明了Hamilton-Jacobi-Bellman型和Pontryagin型的最优性条件。这些平均场结果反映了学习问题的概率性质。此外,通过诉诸平均场庞特里亚金的最大值原理,我们建立了人口和经验学习问题之间的一些定量关系。这为研究最优控制和深度学习之间的算法和理论联系奠定了数学基础。

英文摘要:

Recent work linking deep neural networks and dynamical systems opened up new avenues to analyze deep learning. In particular, it is observed that new insights can be obtained by recasting deep learning as an optimal control problem on difference or differential equations. However, the mathematical aspects of such a formulation have not been systematically explored. This paper introduces the mathematical formulation of the population risk minimization problem in deep learning as a mean-field optimal control problem. Mirroring the development of classical optimal control, we state and prove optimality conditions of both the Hamilton-Jacobi-Bellman type and the Pontryagin type. These mean-field results reflect the probabilistic nature of the learning problem. In addition, by appealing to the mean-field Pontryagin's maximum principle, we establish some quantitative relationships between population and empirical learning problems. This serves to establish a mathematical foundation for investigating the algorithmic and theoretical connections between optimal control and deep learning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑