通过核多样性增强贝叶斯优化与主动学习
Enhancing Bayesian Optimization and Active Learning Through Kernel Diversity
- University of Georgia（佐治亚大学）
- University of California San Diego（加利福尼亚大学圣迭戈分校）
机构由 AI 辅助整理，请以论文原文为准。
中文总结 AI 辅助
本文提出KENDO框架，将集成高斯过程与分歧感知获取策略结合，实例化KENDO-BO和KENDO-AL，在单/多目标优化及主动学习任务上，兼具更优性能与更高计算效率。
中文摘要 AI 辅助
超参数选择仍是贝叶斯优化（BO）和贝叶斯主动学习（AL）中的关键挑战，因为模型设定不准确会导致性能欠佳，而更准确的全贝叶斯处理通常依赖计算成本高昂的马尔可夫链蒙特卡洛（MCMC）采样。本文提出了一个统一框架KENDO（Kernel ENsemble Disagreement-aware Operator，即核集成分歧感知算子），将集成高斯过程（EGP）与分歧感知获取策略相结合。核心思路是用核集成和自适应贝叶斯权重替代超参数采样，同时结合分歧感知获取策略。在该统一框架内，我们实例化了用于贝叶斯优化的KENDO-BO和用于贝叶斯主动学习的KENDO-AL，证明二者源于具有任务特定获取目标的共同自校正机制。我们还通过保留单优化器条件结构的随机标量化，将该方法扩展至多目标优化。针对合成基准和真实世界基准在单目标优化、多目标优化及主动学习上的全面数值测试表明：（i）KENDO-BO与最先进方法相比具有相当或更优的优化性能，同时计算开销最多降低5倍；（ii）KENDO-AL相较于基于MCMC的主动学习基线实现了更优的预测校准，加速比最高达27倍。
英文摘要
Hyperparameter selection remains a key challenge in Bayesian optimization (BO) and Bayesian active learning (AL), as model misspecification can lead to suboptimal performance, while more accurate fully Bayesian treatments typically rely on computationally expensive MCMC sampling. This paper proposes a unified framework, KENDO (Kernel ENsemble Disagreement-aware Operator), that integrates Ensemble Gaussian Processes (EGP) with disagreement-aware acquisition strategies. The central idea is to replace hyperparameter sampling with a kernel ensemble and adaptive Bayesian weighting, combined with disagreement-aware acquisition strategies. Within this unified framework, we instantiate KENDO-BO for BO and KENDO-AL for Bayesian AL, demonstrating that both arise from a common self-correcting mechanism with task-specific acquisition objectives. We further extend the approach to multi-objective optimization via random scalarization that preserves the single-optimizer conditioning structure. Thorough numerical tests on synthetic and real-world benchmarks across single-objective optimization, multi-objective optimization, and active learning demonstrate that (i) KENDO-BO achieves competitive or superior optimization performance compared to state-of-the-art methods while reducing computational overhead by up to $5\times$ and (ii) KENDO-AL achieves superior predictive calibration over MCMC-based active learning baselines with up to $27\times$ speedup.
