DANTE:用于训练神经网络的深度AlterNations
DANTE: Deep AlterNations for Training nEural networks
摘要:
我们提出了DANTE,一种新的方法训练神经网络使用交替最小化原则。DANTE为传统的基于梯度的反向传播技术提供了另一种视角,该技术通常用于训练深度网络。它利用拟凸性的自适应性将训练神经网络转化为双拟凸优化问题。我们表明,对于具有可微(例如sigmoid)和不可微(例如ReLU)激活函数的神经网络配置,我们可以在此公式中有效地执行交替。DANTE还可以扩展到具有多个隐藏层的网络。在标准数据集上的实验中,发现使用所提出的方法训练的神经网络在解决方案的质量以及训练速度方面都比传统的反向传播技术有前途和竞争力。
英文摘要:
We present DANTE, a novel method for training neural networks using the alternating minimization principle. DANTE provides an alternate perspective to traditional gradient-based backpropagation techniques commonly used to train deep networks. It utilizes an adaptation of quasi-convexity to cast training a neural network as a bi-quasi-convex optimization problem. We show that for neural network configurations with both differentiable (e.g. sigmoid) and non-differentiable (e.g. ReLU) activation functions, we can perform the alternations effectively in this formulation. DANTE can also be extended to networks with multiple hidden layers. In experiments on standard datasets, neural networks trained using the proposed method were found to be promising and competitive to traditional backpropagation techniques, both in terms of quality of the solution, as well as training speed.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。