arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1707.06541stat.MLcs.AIcs.LGmath.OCmath.PR
收藏

贝叶斯优化的无离散知识梯度方法

Discretization-free Knowledge Gradient Methods for Bayesian Optimization

Jian Wu, Peter I. Frazier

摘要:

研究了先验信息相关且连续域的贝叶斯排序与选择问题,即贝叶斯优化问题。知识梯度方法[Frazier等人,2008年,2009年]已被广泛研究的离散R&S问题,抽样的一步贝叶斯最优点。当在连续域上使用时,先前关于知识梯度的工作[Scott等人,2011,Wu和Frazier,2016,Wu等人,[2017]通常依赖于离散有限近似。然而,离散化引入了误差,并且随着域的维数的增加,尺度变差。本文提出了一种求解贝叶斯优化的快速无离散化知识梯度方法。我们的方法并不局限于完全顺序设置,但在所有设置中,知识梯度可以使用连续域有用。我们展示了如何将我们的方法推广到处理(i)批量点建议(并行知识梯度);(ii)在优化过程中可获得衍生信息的设置(启用衍生知识梯度)。在数值实验中,我们证明了离散化自由知识梯度方法找到全局最优解显着快于以前的贝叶斯优化算法的合成测试功能和现实世界的应用,特别是当功能评估是嘈杂的;和导数启用知识梯度可以进一步提高性能,甚至优于基于梯度的优化器,如BFGS导数信息可用。

英文摘要:

This paper studies Bayesian ranking and selection (R&S) problems with correlated prior beliefs and continuous domains, i.e. Bayesian optimization (BO). Knowledge gradient methods [Frazier et al., 2008, 2009] have been widely studied for discrete R&S problems, which sample the one-step Bayes-optimal point. When used over continuous domains, previous work on the knowledge gradient [Scott et al., 2011, Wu and Frazier, 2016, Wu et al., 2017] often rely on a discretized finite approximation. However, the discretization introduces error and scales poorly as the dimension of domain grows. In this paper, we develop a fast discretization-free knowledge gradient method for Bayesian optimization. Our method is not restricted to the fully sequential setting, but useful in all settings where knowledge gradient can be used over continuous domains. We show how our method can be generalized to handle (i) batch of points suggestion (parallel knowledge gradient); (ii) the setting where derivative information is available in the optimization process (derivative-enabled knowledge gradient). In numerical experiments, we demonstrate that the discretization-free knowledge gradient method finds global optima significantly faster than previous Bayesian optimization algorithms on both synthetic test functions and real-world applications, especially when function evaluations are noisy; and derivative-enabled knowledge gradient can further improve the performances, even outperforming the gradient-based optimizer such as BFGS when derivative information is available.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑