arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2004.00974cs.LGcs.CVstat.ML
收藏

Deep-n-Cheap:一个用于低复杂度深度学习的自动搜索框架

Deep-n-Cheap: An Automated Search Framework for Low Complexity Deep Learning

University of Southern California(南加州大学)

Sourya Dey, Saikrishna C. Kanala, Keith M. Chugg, Peter A. Beerel

摘要:

我们介绍了Deep-n-Cheap --一个用于搜索深度学习模型的开源AutoML框架。该搜索包括架构和训练超参数,并支持卷积神经网络和多层感知器。我们的框架是针对部署在基准和自定义数据集,因此,提供了更大程度的搜索空间可定制性相比,更有限的搜索,只有预先存在的模型,从文献。我们还介绍了“搜索转移”的技术,它展示了我们的框架发现的模型对多个数据集的泛化能力。Deep-n-Cheap包括用户可自定义的复杂性惩罚,它可以用训练时间或参数数量来权衡性能。具体来说,我们的框架导致模型提供与最先进的性能相当的性能,而训练时间比其他AutoML和模型搜索框架的模型少1-2个数量级。此外,这项工作调查和发展有关搜索过程的各种见解。特别是,我们展示了贪婪策略的优越性,并证明了我们选择贝叶斯优化作为随机/网格搜索的主要搜索方法。

英文摘要:

We present Deep-n-Cheap -- an open-source AutoML framework to search for deep learning models. This search includes both architecture and training hyperparameters, and supports convolutional neural networks and multi-layer perceptrons. Our framework is targeted for deployment on both benchmark and custom datasets, and as a result, offers a greater degree of search space customizability as compared to a more limited search over only pre-existing models from literature. We also introduce the technique of 'search transfer', which demonstrates the generalization capabilities of the models found by our framework to multiple datasets. Deep-n-Cheap includes a user-customizable complexity penalty which trades off performance with training time or number of parameters. Specifically, our framework results in models offering performance comparable to state-of-the-art while taking 1-2 orders of magnitude less time to train than models from other AutoML and model search frameworks. Additionally, this work investigates and develops various insights regarding the search process. In particular, we show the superiority of a greedy strategy and justify our choice of Bayesian optimization as the primary search methodology over random / grid search.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑