arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2106.06477math.OC
收藏

多层网络optimization中避免局部极小的增量训练方法

Avoiding local minima in multilayer network optimization by incremental training

Alberto De Santis, Giampaolo Liuzzi, Stefano Lucidi, Edoardo Maria Tronci

摘要:

训练一个大型的多层神经网络会由于大量无用的固定点而带来许多困难。这些点通常会吸引在训练阶段使用的最小化算法,因此导致效率低下。扩展了文献中针对浅网络提出的一些结果,我们提出了在深度神经网络训练中出现的一类此类稳定点的数学表征。通过这样的描述,我们能够定义一个增量训练算法,避免陷入这些不期望的固定点的吸引区域。

英文摘要:

Training a large multilayer neural network can present many difficulties due to the large number of useless stationary points. These points usually attract the minimization algorithm used during the training phase, which therefore results inefficient. Extending some results proposed in literature for shallow networks, we propose the mathematical characterization of a class of such stationary points that arise in deep neural networks training. Availing such a description, we are able to define an incremental training algorithm that avoids getting stuck in the region of attraction of these undesirable stationary points.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑