深度残差网络的全局收敛多级训练
Globally Convergent Multilevel Training of Deep Residual Networks
摘要：
我们提出了一种全局收敛的深度残差网络多级训练方法（ResNets）。所设计的方法可以被看作是一种新的变体的递归多级信赖域（RMTR）的方法，它在混合（随机确定性）设置自适应调整小批量大小在训练过程中。多层层次结构和传输算子是通过利用动力系统的观点来构造的，该观点将通过ResNet的前向传播解释为初始值问题的前向欧拉离散化。与传统的训练方法相比，我们的新RMTR方法还采用了有限内存SR1方法的多层层次结构的所有级别上的曲率信息。我们的多级训练方法的整体性能和收敛性能进行了数值研究，使用分类和回归领域的例子。
英文摘要：
We propose a globally convergent multilevel training method for deep residual networks (ResNets). The devised method can be seen as a novel variant of the recursive multilevel trust-region (RMTR) method, which operates in hybrid (stochastic-deterministic) settings by adaptively adjusting mini-batch sizes during the training. The multilevel hierarchy and the transfer operators are constructed by exploiting a dynamical system's viewpoint, which interprets forward propagation through the ResNet as a forward Euler discretization of an initial value problem. In contrast to traditional training approaches, our novel RMTR method also incorporates curvature information on all levels of the multilevel hierarchy by means of the limited-memory SR1 method. The overall performance and the convergence properties of our multilevel training method are numerically investigated using examples from the field of classification and regression.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
