arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2003.10870cs.LGstat.ML
收藏

成本感知贝叶斯优化

Cost-aware Bayesian Optimization

Cornell University(康奈尔大学);Amazon, Berlin, Germany(亚马逊柏林分公司)

Eric Hans Lee, Valerio Perrone, Cedric Archambeau, Matthias Seeger

摘要:

贝叶斯优化(BO)是一类全局优化算法,适用于在尽可能少的函数评估中最小化昂贵的目标函数。虽然BO预算通常在迭代中给出,但这隐含地根据迭代计数来度量收敛,并假设每个评估具有相同的成本。实际上,评估成本在搜索空间的不同区域中可能不同。例如,神经网络训练的成本与层大小成二次方增加,这是一个典型的超参数。成本感知的BO测量与时间、精力或金钱等替代成本指标的融合,普通BO方法不适合这些成本指标。我们引入成本分摊BO(CArBO),它试图以尽可能少的成本最小化目标函数。CArBO结合了成本效益的初始设计与成本冷却优化阶段,随着迭代的进行,该阶段会使学习到的成本模型贬值。在一组20黑盒函数优化问题,我们表明,在相同的成本预算,CArBO发现显着更好的超参数配置比竞争的方法。

英文摘要:

Bayesian optimization (BO) is a class of global optimization algorithms, suitable for minimizing an expensive objective function in as few function evaluations as possible. While BO budgets are typically given in iterations, this implicitly measures convergence in terms of iteration count and assumes each evaluation has identical cost. In practice, evaluation costs may vary in different regions of the search space. For example, the cost of neural network training increases quadratically with layer size, which is a typical hyperparameter. Cost-aware BO measures convergence with alternative cost metrics such as time, energy, or money, for which vanilla BO methods are unsuited. We introduce Cost Apportioned BO (CArBO), which attempts to minimize an objective function in as little cost as possible. CArBO combines a cost-effective initial design with a cost-cooled optimization phase which depreciates a learned cost model as iterations proceed. On a set of 20 black-box function optimization problems we show that, given the same cost budget, CArBO finds significantly better hyperparameter configurations than competing methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑