arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2206.04771cs.LGstat.ML
收藏

最大信息贝叶斯优化的联合熵搜索

Joint Entropy Search for Maximally-Informed Bayesian Optimization

Lund University(隆德大学);University of Freiburg(弗赖堡大学);Bosch Center for Artificial Intelligence(博世人工智能中心);Stanford University(斯坦福大学);DBtune(DBtune公司)

Carl Hvarfner, Frank Hutter, Luigi Nardi

摘要:

信息理论贝叶斯优化技术已经成为流行的优化昂贵的评估黑盒功能,由于其非近视的品质。熵搜索和预测熵搜索都考虑了输入空间中最优值上的熵,而最近的最大值熵搜索考虑了输出空间中最优值上的熵。我们提出了联合熵搜索(JES),一种新的信息理论的采集功能,认为一个全新的量,即在输入和输出空间的联合最佳概率密度熵。为了将这些信息,我们考虑减少熵条件幻想的最佳输入/输出对。由此产生的方法主要依赖于标准的GP机器,并消除了通常与信息理论方法相关的复杂近似。以最小的计算开销,JES显示出卓越的决策能力,并为跨各种任务的信息理论方法提供最先进的性能。作为一个轻量级的方法与优越的结果,JES提供了一个新的去获取功能的贝叶斯优化。

英文摘要:

Information-theoretic Bayesian optimization techniques have become popular for optimizing expensive-to-evaluate black-box functions due to their non-myopic qualities. Entropy Search and Predictive Entropy Search both consider the entropy over the optimum in the input space, while the recent Max-value Entropy Search considers the entropy over the optimal value in the output space. We propose Joint Entropy Search (JES), a novel information-theoretic acquisition function that considers an entirely new quantity, namely the entropy over the joint optimal probability density over both input and output space. To incorporate this information, we consider the reduction in entropy from conditioning on fantasized optimal input/output pairs. The resulting approach primarily relies on standard GP machinery and removes complex approximations typically associated with information-theoretic methods. With minimal computational overhead, JES shows superior decision-making, and yields state-of-the-art performance for information-theoretic approaches across a wide suite of tasks. As a light-weight approach with superior results, JES provides a new go-to acquisition function for Bayesian optimization.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑