这也终将过去:动态贝叶斯优化中移除陈旧观测
This Too Shall Pass: Removing Stale Observations in Dynamic Bayesian Optimization
AI总结:
本文针对动态贝叶斯优化中陈旧观测降低跟踪效率的问题,提出基于Wasserstein距离的观测相关性准则及W-DBO算法,通过在线移除不相关观测,在未知时间范围的连续优化任务中同时保持高预测性能和高采样频率,并实验验证其显著优于现有方法。
摘要:
贝叶斯优化(BO)已被证明在优化静态、带噪声、评估代价高昂的黑箱函数 $f : \mathcal{S} \to \mathbb{R}$ 方面非常成功。然而,优化一个同时是时间函数的黑箱(即动态函数)$f : \mathcal{S} \times \mathcal{T} \to \mathbb{R}$ 仍然是一个挑战,因为动态贝叶斯优化(DBO)算法必须随时间持续跟踪最优值。这至少在三个方面改变了优化问题的性质:(i)在 $\mathcal{S} \times \mathcal{T}$ 中任意查询一个点是不可能的,(ii)随着时间推移,过去的观测对于跟踪最优值变得越来越不相关,以及(iii)DBO 算法必须具有较高的采样频率,以便收集足够多的相关观测来随时间跟踪最优值。在本文中,我们设计了一个基于 Wasserstein 距离的准则,能够量化一个观测相对于未来预测的相关性。然后,我们利用该准则构建了 W-DBO,这是一种能够即时从数据集中移除不相关观测的 DBO 算法,从而同时保持良好的预测性能和高采样频率,即使在时间范围未知的连续时间优化任务中也是如此。数值实验确立了 W-DBO 的优越性,它以明显的优势超越了当前最先进的方法。
英文摘要:
Bayesian Optimization (BO) has proven to be very successful at optimizing a static, noisy, costly-to-evaluate black-box function $f : \mathcal{S} \to \mathbb{R}$. However, optimizing a black-box which is also a function of time (i.e., a dynamic function) $f : \mathcal{S} \times \mathcal{T} \to \mathbb{R}$ remains a challenge, since a dynamic Bayesian Optimization (DBO) algorithm has to keep track of the optimum over time. This changes the nature of the optimization problem in at least three aspects: (i) querying an arbitrary point in $\mathcal{S} \times \mathcal{T}$ is impossible, (ii) past observations become less and less relevant for keeping track of the optimum as time goes by and (iii) the DBO algorithm must have a high sampling frequency so it can collect enough relevant observations to keep track of the optimum through time. In this paper, we design a Wasserstein distance-based criterion able to quantify the relevancy of an observation with respect to future predictions. Then, we leverage this criterion to build W-DBO, a DBO algorithm able to remove irrelevant observations from its dataset on the fly, thus maintaining simultaneously a good predictive performance and a high sampling frequency, even in continuous-time optimization tasks with unknown horizon. Numerical experiments establish the superiority of W-DBO, which outperforms state-of-the-art methods by a comfortable margin.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。