arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2003.13826cs.NE
收藏

初始设计策略及其对顺序模型优化的影响

Initial Design Strategies and their Effects on Sequential Model-Based Optimization

Jakob Bossek, Carola Doerr, Pascal Kerschke

摘要:

序贯基于模型的优化(SMBO)方法是用于解决需要计算或以其他方式昂贵的函数评估的问题的算法。SMBO的关键设计原则是用代理替代真实的目标函数,该代理用于提出接下来要评估的点。SMBO算法本质上是模块化的,给用户留下了许多重要的设计选择。重要的研究工作是了解哪种设置最适合哪种类型的问题。然而,大多数作品都集中在模型的选择,采集功能,以及用于优化后者的策略。然而,初始采样策略的选择受到的关注要少得多。毫不奇怪,在文献中可以找到相当不同的建议。在这项工作中,我们分析了初始样本的大小和分布如何影响有效的全局优化~(EGO)算法,一个著名的SMBO方法的整体质量。虽然总体而言,使用Halton抽样的小初始预算似乎更可取,但我们也观察到性能前景相当不稳定。此外,我们确定了几种情况下,EGO执行不利的随机抽样。这两个观察表明,自适应SMBO设计可能是有益的,使SMBO自动算法设计的一个有趣的测试床。

英文摘要:

Sequential model-based optimization (SMBO) approaches are algorithms for solving problems that require computationally or otherwise expensive function evaluations. The key design principle of SMBO is a substitution of the true objective function by a surrogate, which is used to propose the point(s) to be evaluated next. SMBO algorithms are intrinsically modular, leaving the user with many important design choices. Significant research efforts go into understanding which settings perform best for which type of problems. Most works, however, focus on the choice of the model, the acquisition function, and the strategy used to optimize the latter. The choice of the initial sampling strategy, however, receives much less attention. Not surprisingly, quite diverging recommendations can be found in the literature. We analyze in this work how the size and the distribution of the initial sample influences the overall quality of the efficient global optimization~(EGO) algorithm, a well-known SMBO approach. While, overall, small initial budgets using Halton sampling seem preferable, we also observe that the performance landscape is rather unstructured. We furthermore identify several situations in which EGO performs unfavorably against random sampling. Both observations indicate that an adaptive SMBO design could be beneficial, making SMBO an interesting test-bed for automated algorithm design.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑