arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2204.08528math.OCcs.LGcs.NAmath.NA
收藏

一种用于深度神经网络的最优时变学习框架

An Optimal Time Variable Learning Framework for Deep Neural Networks

George Mason University(乔治梅森大学);University of Delaware(特拉华大学)

Harbir Antil, Hugo Díaz, Evelyn Herberg

摘要:

深度神经网络(DNN)中的特征传播可以与非线性离散动力系统相关联。新颖之处,在本文中,在于让离散化参数(时间步长)从一层到另一层,这需要学习,在一个优化框架。所提出的框架可以应用于任何现有的网络,如ResNet,DenseNet或Fractional-DNN。该框架有助于克服消失和爆炸梯度问题。一些现有的连续DNN,如分数DNN的稳定性也进行了研究。所提出的方法适用于不适定的3D-Maxwell方程。

英文摘要:

Feature propagation in Deep Neural Networks (DNNs) can be associated to nonlinear discrete dynamical systems. The novelty, in this paper, lies in letting the discretization parameter (time step-size) vary from layer to layer, which needs to be learned, in an optimization framework. The proposed framework can be applied to any of the existing networks such as ResNet, DenseNet or Fractional-DNN. This framework is shown to help overcome the vanishing and exploding gradient issues. Stability of some of the existing continuous DNNs such as Fractional-DNN is also studied. The proposed approach is applied to an ill-posed 3D-Maxwell's equation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑