arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2607.04356cs.LG
收藏

贝叶斯优化需要多少初始点?

How Many Initial Points Does Bayesian Optimization Need?

Korea Advanced Institute of Science & Technology (KAIST)(韩国科学技术院);HayanMind Inc.(哈扬思维公司);Technical University of Nuremberg (UTN)(纽伦堡工业大学);Munich Center for Machine Learning (MCML)(慕尼黑机器学习中心);Helmholtz Munich(亥姆霍兹慕尼黑中心);Imperial College London(伦敦帝国理工学院)

Mujin Cheon, James Odgers, Dong-Yeun Koh, Calvin Tsay

AI总结:

研究贝叶斯优化初始点数量选择问题,发现初始点数量与总成本呈U型关系,不同超参数和采集函数都存在此权衡,汤普森采样是例外,还给出实用建议。

摘要:

贝叶斯优化(BO)通常始于初始化阶段:一批\(n_0\)个无信息评估。\(n_0\)的选择很大程度上仍是启发式的,我们通过实证观察到总成本(随机初始点加上找到全局最优所需的BO迭代次数)在\(n_0\)上呈U型,即选择过低或过高的\(n_0\)值都会浪费资源。我们发现这种权衡在最大似然估计、贝叶斯马尔可夫链蒙特卡罗和精确高斯过程超参数以及采集函数中都存在。对于后者,汤普森采样似乎是个例外,总成本和简单遗憾基本上与\(n_0\)无关,尽管在我们的实验中更高。我们将这种U型归因于方差驱动的BO已知的边界问题:BO在转向内部之前会在超立方体的角落消耗早期预算。我们使用已知精确超参数的3D BO轨迹展示了这种效果。我们最后给出实用建议:尽可能使用多步前瞻BO;否则,当无法调整\(n_0\)时使用汤普森采样,当可以调整时使用非常大的\(n_0\)。

英文摘要:

Bayesian Optimization (BO) generally begins with an initialization phase: a batch of $n_0$ uninformed evaluations. The choice of $n_0$ remains largely heuristic, and we empirically observe that the total cost (random initial points plus BO iterations needed to find the global optimum) is U-shaped in $n_0$, i.e., a practitioner wastes resources by selecting either too low or too high a value of $n_0$. We find this tradeoff persists across MLE, Bayesian MCMC, and exact GP hyperparameters, as well as across acquisition functions. Toward the latter, Thompson Sampling appears an exception, with both total cost and simple regret essentially $n_0$-agnostic, though higher in our experiments. We attribute this U-shape to the known boundary issue of variance-driven BO: BO burns early budget on corners of the hypercube before turning inward. We demonstrate this effect using a 3D BO trajectory where the exact hyperparameters are known. We conclude with practical recommendations: use multi-step lookahead BO where possible; otherwise use Thompson Sampling when $n_0$ cannot be tuned, and a generously large $n_0$ when it can.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑