How Many Initial Points Does Bayesian Optimization Need?
Korea Advanced Institute of Science & Technology (KAIST)(韩国科学技术院);HayanMind Inc.(哈扬思维公司);Technical University of Nuremberg (UTN)(纽伦堡工业大学);Munich Center for Machine Learning (MCML)(慕尼黑机器学习中心);Helmholtz Munich(亥姆霍兹慕尼黑中心);Imperial College London(伦敦帝国理工学院)
Mujin Cheon, James Odgers, Dong-Yeun Koh, Calvin Tsay
备注:
4 pages. Accepted at the ICML 2026 Workshop on Decision-Making from Offline Datasets to Online Adaptation
Bayesian Optimization (BO) generally begins with an initialization phase: a batch of $n_0$ uninformed evaluations. The choice of $n_0$ remains largely heuristic, and we empirically observe that the total cost (random initial points plus BO iterations needed to find the global optimum) is U-shaped in $n_0$, i.e., a practitioner wastes resources by selecting either too low or too high a value of $n_0$. We find this tradeoff persists across MLE, Bayesian MCMC, and exact GP hyperparameters, as well as across acquisition functions. Toward the latter, Thompson Sampling appears an exception, with both total cost and simple regret essentially $n_0$-agnostic, though higher in our experiments. We attribute this U-shape to the known boundary issue of variance-driven BO: BO burns early budget on corners of the hypercube before turning inward. We demonstrate this effect using a 3D BO trajectory where the exact hyperparameters are known. We conclude with practical recommendations: use multi-step lookahead BO where possible; otherwise use Thompson Sampling when $n_0$ cannot be tuned, and a generously large $n_0$ when it can.
OASI: Objective-Aware Surrogate Initialization for Multi-Objective Bayesian Optimization in TinyML Keyword Spotting
Department of Electronics and Communication Engineering, National Institute of Technology Durgapur(电子与通信工程系,德里帕尔国家理工学院);System Research and Applications Department, STMicroelectronics(系统研究与应用部,STMicroelectronics)
Default Machine Learning Hyperparameters Do Not Provide Informative Initialization for Bayesian Optimization
Department of Telematics and Computing, Universidad Pontificia Comillas(电信与计算系,Pontificia Comillas大学);Institute for Research in Technology (IIT), Universidad Pontificia Comillas(技术研究 institute,Pontificia Comillas大学)
How does Bayesian Sampling help Membership Inference Attacks?
Department of Statistics and Data Science, Southern University of Science and Technology(统计与数据科学系,南方科技大学);Shanghai Innovation Institute(上海创新研究院);School of Computer Science, Nanjing University(南京大学计算机科学系);Center for Applied Statistics and School of Statistics, Renmin University of China(应用统计中心和统计学系,中国人民大学)