AutoLR:一种学习率策略的演化方法
AutoLR: An Evolutionary Approach to Learning Rate Policies
摘要:
选择合适的学习率对于良好的人工神经网络训练和性能至关重要。在过去,人们不得不依靠经验和试错来找到足够的学习率。目前,存在过多的现有技术的自动方法,使得搜索良好的学习率更容易。虽然这些技术是有效的,多年来取得了良好的效果,但它们是通用的解决方案。这意味着特定网络拓扑结构的学习率优化在很大程度上仍未得到探索。这项工作提出了AutoLR,一个框架,使用结构化语法进化为特定的神经网络架构进化学习率参数。该系统被用来发展学习率的政策,比较常用的基线值学习率。结果表明,使用某些进化策略进行的训练比建立的基线更有效,并表明这种方法是提高神经网络性能的可行方法。
英文摘要:
The choice of a proper learning rate is paramount for good Artificial Neural Network training and performance. In the past, one had to rely on experience and trial-and-error to find an adequate learning rate. Presently, a plethora of state of the art automatic methods exist that make the search for a good learning rate easier. While these techniques are effective and have yielded good results over the years, they are general solutions. This means the optimization of learning rate for specific network topologies remains largely unexplored. This work presents AutoLR, a framework that evolves Learning Rate Schedulers for a specific Neural Network Architecture using Structured Grammatical Evolution. The system was used to evolve learning rate policies that were compared with a commonly used baseline value for learning rate. Results show that training performed using certain evolved policies is more efficient than the established baseline and suggest that this approach is a viable means of improving a neural network's performance.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。