Why to "grow" and "harvest" deep learning models?
Samara University（萨马拉大学）；Institute of Applied Ecology, Oikon Ltd.（应用生态学研究所，Oikon有限公司）
英文摘要：
Current expectations from training deep learning models with gradient-based methods include: 1) transparency; 2) high convergence rates; 3) high inductive biases. While the state-of-art methods with adaptive learning rate schedules are fast, they still fail to meet the other two requirements. We suggest reconsidering neural network models in terms of single-species population dynamics where adaptation comes naturally from open-ended processes of "growth" and "harvesting". We show that the stochastic gradient descent (SGD) with two balanced pre-defined values of per capita growth and harvesting rates outperform the most common adaptive gradient methods in all of the three requirements.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
