发表机构
Vector Institute; INRIA Saclay; University of Waterloo; Waterloo Institute for Nanotechnology; Western University（向量研究所; 法国国家信息与自动化研究所萨克雷研究中心; 滑铁卢大学; 滑铁卢纳米技术研究所; 西安大略大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出ELF-BO算法，利用评估延迟并行计算建议，实现全贝叶斯优化，在匹配性能的同时降低决策延迟，使全贝叶斯优化实用化。
AI 中文摘要
黑盒优化问题在科学和工程领域无处不在，通常涉及昂贵的客观函数。这种客观延迟在优化过程中产生两个后果：（i）客观评估主导执行时间，（ii）样本高效的算法对于加速开发和避免资源浪费至关重要。贝叶斯优化（BO）方法是规划者提出下一个尝试点的事实上的选择。标准BO使用点估计来拟合代理模型的超参数。或者，全贝叶斯方法使用模型平均来考虑超参数的不确定性，从而获得更好的不确定性估计——这在BO中普遍存在的低数据场景中很有用。然而，这种方法通常成本过高，因此很少使用。在这项工作中，我们提出了ELF-BO，一种利用客观评估延迟来提前计算下一个建议的算法，使得全贝叶斯优化在不产生显著决策时间成本的情况下得以实现。这是通过在客观评估期间从超参数后验中采样来实现的，仅在观察到客观值后需要对样本进行重新加权。在合成函数和现实世界应用中，我们展示了ELF-BO在匹配全贝叶斯方法性能的同时，仅产生与标准BO相当或更好的决策延迟。因此，ELF-BO使全贝叶斯优化在实际用例中变得实用。
英文摘要
Black-box optimization problems are ubiquitous across science and engineering, often dealing with expensive objective functions. This objective latency has two consequences during optimization: (i) the objective evaluation dominates execution time, and (ii) sample-efficient algorithms are crucial to accelerate development and avoid wasting resources. Bayesian optimization (BO) methods are the \textit{de facto} choice of planners for suggesting the next point to try. Standard BO fits the surrogate model's hyperparameters with a point estimate. Alternatively, a fully Bayesian approach uses model averaging to account for uncertainty over the hyperparameters, leading to better uncertainty estimates---useful in the low-data regime that is pervasive in BO. However, it is often prohibitively expensive and thus rarely used. In this work, we propose ELF-BO, an algorithm that uses the objective evaluation latency to headstart the computation of the next suggestion, allowing for fully Bayesian optimization without incurring substantial decision-time costs. This is done by sampling from the hyperparameter posterior \emph{while} the objective is being evaluated, only requiring reweighting of the samples once the objective value is observed. Across synthetic functions and real-world applications, we show that ELF-BO matches the performance of fully Bayesian methods while only incurring decision latency on par with or better than standard BO. Thus, ELF-BO makes fully Bayesian optimization practical in real-world use cases.
