具有有限库输入扭曲核的无遗憾贝叶斯优化
No-Regret Bayesian Optimization with Finite-Library Input-Warped Kernels
- AI Sweden（瑞典人工智能研究院）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本文提出FLIWBO方法，可从有限库选输入扭曲，在 mild 假设下保收敛保证，在多基准测试中优于GP-UCB，能修复几何不匹配、逃脱陷阱，在高代价MAS设计中可行。
AI中文摘要：
高斯过程贝叶斯优化（GP-BO）在黑箱优化高代价函数（如超参数优化（HPO）和多智能体系统（MAS）设计）中表现出色。部分方法（尤其是高斯过程上置信界（GP-UCB））具有收敛率保证，但需要固定核。关键在于，核编码了输入邻近度对目标值相似性的影响。当原始坐标与该几何结构不匹配时（如对数缩放超参数或局部峰值），输入扭曲可大幅提升样本效率，但已知的GP-UCB证明要求核固定。我们提出有限库输入扭曲贝叶斯优化（FLIWBO），可通过任意依赖历史的规则从平滑输入映射的有限库中选择扭曲方式。该方法在 mild 假设下保留高概率收敛保证，同时适应输入几何以加速学习，具有显式的√(N_ε)库规模成本。受控诊断显示，有限库扭曲可修复植入的几何不匹配并识别FLIWBO的失败案例。在四个重复基准测试（扭曲合成目标、置信围栏陷阱、Fashion-MNIST HPO）中，FLIWBO-UCB在几何误配情况下优于原始坐标GP-UCB，可逃脱甚至最优扭曲期望改进也无法应对的陷阱，恢复了手动对数缩放带来的大部分增益，同时引领了所有具有匹配遗憾保证的测试方法。一项20维MAS设计研究进一步证明其在高代价噪声评估下的可行性。实验代码可访问：this https URL。
英文摘要：
Gaussian-process Bayesian optimization (GP-BO) excels at black-box optimization of costly functions, e.g., hyperparameter optimization (HPO) and multi-agent system (MAS) design. Convergence-rate guarantees exist for select methods, notably GP upper confidence bound (GP-UCB), but require a fixed kernel. Critically, the kernel encodes how input proximity affects objective value similarity. When raw coordinates poorly match this geometry - as with log-scaled hyperparameters or localized peaks - input warping can greatly improve sample efficiency, yet known GP-UCB proofs require a fixed kernel. We propose Finite-Library Input-Warped Bayesian Optimization (FLIWBO), which selects warps from a finite library of smooth input maps by any history-dependent rule. It adapts the input geometry to accelerate learning while retaining high-probability convergence guarantees under mild hypotheses, with an explicit $\sqrt(N_\varepsilon)$ library-size cost. Controlled diagnostics show that finite-library warping repairs planted geometry mismatches and identify FLIWBO failure cases. Across four repeated benchmarks - warped synthetic objectives, a confidence-fence trap, and Fashion-MNIST HPO - FLIWBO-UCB beats raw-coordinate GP-UCB under misspecified geometry, escapes traps that defeat even oracle-warp expected improvement, and recovers much of the gain from manual log scaling, while leading the tested methods that admit a matching regret guarantee. A 20-dimensional MAS design study further shows feasibility under costly noisy evaluations. Code for experiments is available: https://github.com/edvin-ketabati/bogp-paper-experiments.
