BILBO:双层贝叶斯优化
BILBO: BILevel Bayesian Optimization
AI总结:
提出 BILBO 算法,通过基于置信边界的可信集和单次查询策略同时优化双层黑盒问题,避免了重复下层优化,并在理论与实验上均验证了其有效性。
摘要:
双层优化的特征在于一种两层的优化结构,其中上层问题受限于最优的下层解,这种结构在现实世界的问题中普遍存在。受限于最优下层解带来了重大挑战,特别是在有噪声、有约束和无导数的设置中,因为重复的下层优化样本效率低下,且预测的下层解可能是次优的。我们提出了 BILBO(双层贝叶斯优化),一种用于具有黑盒函数的一般双层问题的新型贝叶斯优化算法,它同时优化上层和下层问题,而无需现有方法所需的重复的下层优化。BILBO 从基于置信边界的可信集合中采样,这限制了下层次优性的界限。此外,BILBO 每次迭代仅选择一个函数查询,其中函数查询选择策略结合了估计下层解的不确定性,并包含查询的条件重新分配,以鼓励对下层目标的探索。BILBO 的性能在理论上得到了保证,对于常用的核具有次线性的遗憾界限,并在几个合成和现实世界的问题上进行了经验评估。
英文摘要:
Bilevel optimization is characterized by a two-level optimization structure, where the upper-level problem is constrained by optimal lower-level solutions, and such structures are prevalent in real-world problems. The constraint by optimal lower-level solutions poses significant challenges, especially in noisy, constrained, and derivative-free settings, as repeating lower-level optimizations is sample inefficient and predicted lower-level solutions may be suboptimal. We present BILevel Bayesian Optimization (BILBO), a novel Bayesian optimization algorithm for general bilevel problems with blackbox functions, which optimizes both upper- and lower-level problems simultaneously, without the repeated lower-level optimization required by existing methods. BILBO samples from confidence-bounds based trusted sets, which bounds the suboptimality on the lower level. Moreover, BILBO selects only one function query per iteration, where the function query selection strategy incorporates the uncertainty of estimated lower-level solutions and includes a conditional reassignment of the query to encourage exploration of the lower-level objective. The performance of BILBO is theoretically guaranteed with a sublinear regret bound for commonly used kernels and is empirically evaluated on several synthetic and real-world problems.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。