深度神经网络的进化学习率优化算法
Evolving Learning Rate Optimizers for Deep Neural Networks
摘要：
人工神经网络（ANN）由于其在图像识别、语音识别等难题中的成功应用而受到广泛关注.然而，当从业者想要设计ANN时，他们需要经历选择一组参数和拓扑结构的费力过程。目前，有几种最先进的方法允许自动选择这些方面中的一些。学习率优化器是一组搜索学习率的良好值的此类技术。虽然这些技术是有效的，并且多年来取得了良好的效果，但它们是通用的解决方案，即它们不考虑特定网络的特性。我们提出了一个名为AutoLR的框架来自动设计学习率优化器。详细介绍了该系统的两个版本。第一个是动态AutoLR，它根据当前时期和以前的学习率来进化静态和动态学习率优化器。第二个版本，自适应AutoLR，进化自适应优化器，可以微调每个网络eight的学习率，使它们通常更有效。其结果与最先进的方法相比具有竞争力，甚至在某些情况下优于它们。此外，该系统发展了一个分类器，ADES，这似乎是新颖的和创新的，因为据我们所知，它具有不同于最先进的方法的结构。
英文摘要：
Artificial Neural Networks (ANNs) became popular due to their successful application difficult problems such image and speech recognition. However, when practitioners want to design an ANN they need to undergo laborious process of selecting a set of parameters and topology. Currently, there are several state-of-the art methods that allow for the automatic selection of some of these aspects. Learning Rate optimizers are a set of such techniques that search for good values of learning rates. Whilst these techniques are effective and have yielded good results over the years, they are general solution i.e. they do not consider the characteristics of a specific network. We propose a framework called AutoLR to automatically design Learning Rate Optimizers. Two versions of the system are detailed. The first one, Dynamic AutoLR, evolves static and dynamic learning rate optimizers based on the current epoch and the previous learning rate. The second version, Adaptive AutoLR, evolves adaptive optimizers that can fine tune the learning rate for each network eeight which makes them generally more effective. The results are competitive with the best state of the art methods, even outperforming them in some scenarios. Furthermore, the system evolved a classifier, ADES, that appears to be novel and innovative since, to the best of our knowledge, it has a structure that differs from state of the art methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
