CobBO：带两阶段核的协调退避贝叶斯优化
CobBO: Coordinate Backoff Bayesian Optimization with Two-Stage Kernels
摘要：
贝叶斯优化是一种用于优化昂贵的黑盒函数的流行方法。然而，它经常在计算可能过于繁重的高维空间中挣扎。为了缓解这个问题，我们引入了具有两阶段内核的协调退避贝叶斯优化（CobBO）。在每一轮中，第一阶段使用一个简单的粗糙内核，牺牲近似精度以提高计算效率。它通过有意消除局部波动来捕捉全球景观。然后，在同一轮的第二阶段中，将全空间中的过去观测点投影到所选择的子空间以形成虚拟点。这些虚拟点，连同使用第一阶段的简单内核估计的其未知函数值的均值和方差，在第二阶段中被拟合到更复杂的内核模型。在所选择的低维子空间内，在其中进行贝叶斯优化的计算成本变得负担得起。为了进一步提高性能，在同一子空间中收集一系列连续的观测值，这可以有效地改进函数的近似。这种细化一直持续到满足停止规则，确定何时从某个子空间后退并切换到另一个子空间。这种解耦显著降低了高维中的计算负担，这充分利用了整个空间中的观测，而不是仅依赖于每个坐标子空间中的观测。广泛的评估表明，CobBO找到了与其他最先进的方法相当或更好的解决方案，尺寸范围从几十到几百，同时降低了试验的复杂性和计算成本。
英文摘要：
Bayesian optimization is a popular method for optimizing expensive black-box functions. Yet it oftentimes struggles in high dimensions where the computation could be prohibitively heavy. To alleviate this problem, we introduce Coordinate backoff Bayesian Optimization (CobBO) with two-stage kernels. During each round, the first stage uses a simple coarse kernel that sacrifices the approximation accuracy for computational efficiency. It captures the global landscape by purposely smoothing away local fluctuations. Then, in the second stage of the same round, past observed points in the full space are projected to the selected subspace to form virtual points. These virtual points, along with the means and variances of their unknown function values estimated using the simple kernel of the first stage, are fitted to a more sophisticated kernel model in the second stage. Within the selected low dimensional subspace, the computational cost of conducting Bayesian optimization therein becomes affordable. To further enhance the performance, a sequence of consecutive observations in the same subspace are collected, which can effectively refine the approximation of the function. This refinement lasts until a stopping rule is met determining when to back off from a certain subspace and switch to another. This decoupling significantly reduces the computational burden in high dimensions, which fully leverages the observations in the whole space rather than only relying on observations in each coordinate subspace. Extensive evaluations show that CobBO finds solutions comparable to or better than other state-of-the-art methods for dimensions ranging from tens to hundreds, while reducing both the trial complexity and computational costs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
