arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2103.07963math.OCcs.AI
收藏

在超参数optimization中使用静态代理

Use of static surrogates in hyperparameter optimization

GERAD;Polytechnique Montréal(蒙特利尔理工学院)

Dounia Lakhmiri, Sébastien Le Digabel

摘要:

优化神经网络的超参数和架构是开发任何新应用程序的一个漫长而必要的阶段。这种消耗过程可以从精心设计的策略中受益,这些策略旨在快速丢弃低质量配置并专注于更有希望的候选者。这项工作旨在增强HyperNOMAD,这是一个库,它采用直接搜索无导数优化算法来同时调整神经网络的架构和训练,方法是针对其执行的两个关键步骤,并利用静态代理形式的廉价近似来触发配置评估的早期停止和候选池的排名。这些添加到HyperNOMAD被证明可以改善其资源消耗,而不会损害所提出的解决方案的质量。

英文摘要:

Optimizing the hyperparameters and architecture of a neural network is a long yet necessary phase in the development of any new application. This consuming process can benefit from the elaboration of strategies designed to quickly discard low quality configurations and focus on more promising candidates. This work aims at enhancing HyperNOMAD, a library that adapts a direct search derivative-free optimization algorithm to tune both the architecture and the training of a neural network simultaneously, by targeting two keys steps of its execution and exploiting cheap approximations in the form of static surrogates to trigger the early stopping of the evaluation of a configuration and the ranking of pools of candidates. These additions to HyperNOMAD are shown to improve on its resources consumption without harming the quality of the proposed solutions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑