发表机构
University of Wisconsin–Madison（威斯康星大学麦迪逊分校）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对高维贝叶斯优化中ARD GP的过参数化问题，提出Iso-BO用共享长度尺度的各向同性GP替代，在多种基准上常优于现代朴素BO，并保持竞争力。
AI 中文摘要
高维贝叶斯优化（BO）通常从远少于输入维数的观测中拟合高斯过程（GP）代理模型。现代朴素贝叶斯优化在这种情境下可以通过维度感知的先验、初始化和采集优化表现良好，但通常保留自动相关性确定（ARD），即每个输入坐标拟合一个长度尺度。我们研究这种建模选择，并提出Iso-BO，一种受控修改，将ARD GP替换为使用一个共享长度尺度的各向同性GP，同时保持周围的BO流程匹配。对于径向核，我们证明边际对数似然（MLL）对ARD长度尺度的逆平方的依赖仅通过观测输入之间的加权成对距离体现。因此，当前设计可能使某些ARD方向在MLL中完全不可见或仅受弱约束。Iso-BO移除坐标级重新加权，转而拟合一个单一的共享尺度。长度尺度拟合和预测密度诊断表明，这种有限数据效应在实践中出现，包括当数据生成过程是各向异性时。在GP先验、合成和真实世界基准测试中，Iso-BO通常优于匹配的现代朴素贝叶斯优化，并在测试预算下与所包含的高维BO基线保持竞争力。压力测试也显示了预期的边界，即足够强且可学习的各向异性可能有利于更灵活的ARD模型。
英文摘要
High-dimensional Bayesian optimization (BO) often fits Gaussian process (GP) surrogates from far fewer observations than input dimensions. Modern Vanilla BO can perform well in this regime with dimension-aware priors, initialization, and acquisition optimization, but it typically retains automatic relevance determination (ARD), fitting one lengthscale per input coordinate. We study this modeling choice and propose Iso-BO, a controlled modification that replaces the ARD GP with an isotropic GP using one shared lengthscale while keeping the surrounding BO pipeline matched. For radial kernels, we show that the marginal log likelihood (MLL) depends on the inverse-squared ARD lengthscales only through weighted pairwise distances among the observed inputs. The current design can therefore leave some ARD directions exactly invisible or only weakly constrained by the MLL. Iso-BO removes coordinatewise reweighting and fits a single shared scale instead. Lengthscale-fitting and predictive-density diagnostics show that this finite-data effect appears in practice, including when the data-generating process is anisotropic. Across GP-prior, synthetic, and real-world benchmarks, Iso-BO often improves over matched modern Vanilla BO and remains competitive with the included high-dimensional BO baselines under the tested budgets. Stress tests also show the expected boundary wherein sufficiently strong, learnable anisotropy can favor the more flexible ARD model.
