In this paper, we make the key delineation on the roles of resolution and statistical uncertainty in hierarchical bandits-based black-box optimization algorithms, guiding a more general analysis and a more efficient algorithm design. We introduce the \textit{optimum-statistical collaboration}, an algorithm framework of managing the interaction between optimization error flux and statistical error flux evolving in the optimization process. We provide a general analysis of this framework without specifying the forms of statistical error and uncertainty quantifier. Our framework and its analysis, due to their generality, can be applied to a large family of functions and partitions that satisfy different local smoothness assumptions and have different numbers of local optimums, which is much richer than the class of functions studied in prior works. Our framework also inspires us to propose a better measure of the statistical uncertainty and consequently a variance-adaptive algorithm \texttt{VHCT}. In theory, we prove the algorithm enjoys rate-optimal regret bounds under different local smoothness assumptions; in experiments, we show the algorithm outperforms prior efforts in different settings.
Bayesian Optimization with a Prior for the Optimum
Universidade Federal de Minas Gerais(米纳斯吉拉斯联邦大学);Lund University(隆德大学);Stanford University(斯坦福大学);Leibniz University Hannover(莱布尼茨汉诺威大学);University of Freiburg(弗莱堡大学);Bosch Center for Artificial Intelligence(博世人工智能中心)
Department of Computer Science, The University of Manchester(曼彻斯特大学计算机科学系);Department of ISEM, National University of Singapore(新加坡国立大学ISEM系);Alliance Manchester Business School, University of Manchester(曼彻斯特大学阿联酋商业学院)
A Robust Asymmetric Kernel Function for Bayesian Optimization, with Application to Image Defect Detection in Manufacturing Systems
Virginia Tech(弗吉尼亚理工大学);Kuwait University(科威特大学);Department of Industrial and Systems Engineering, Virginia Tech(弗吉尼亚理工大学工业与系统工程系);Department of Industrial and Management Systems Engineering, Kuwait University(科威特大学工业与管理系统工程系)
Some response surface functions in complex engineering systems are usually highly nonlinear, unformed, and expensive-to-evaluate. To tackle this challenge, Bayesian optimization , which conducts sequential design via a posterior distribution over the objective function, is a critical method used to find the global optimum of black-box functions. Kernel functions play an important role in shaping the posterior distribution of the estimated function. The widely used kernel function, e.g., radial basis function (RBF), is very vulnerable and susceptible to outliers; the existence of outliers is ca…
Explaining Bayesian Optimization by Shapley Values Facilitates Human-AI Collaboration
Ludwig-Maximilians-Universität (LMU) Munich(慕尼黑大学);John A. Paulson Harvard School of Engineering and Applied Sciences, Harvard University(哈佛大学约翰·保尔森工程与应用科学学院);Munich Center for Machine Learning (MCML)(慕尼黑机器学习中心);Wyss Institute for Biologically Inspired Engineering, Harvard University(哈佛大学怀斯生物启发工程研究所)
Multi-Objective Multi-Fidelity Bayesian Optimization with Causal Priors
Department of Computer Science;Engineering, University of South Carolina, SC, USA;Computer Science Department, Appalachian State University, NC, USA;Engineering, Texas A\&M University, TX, USA
Microsoft Research(微软研究院);Princeton University(普林斯顿大学)
我们表明,自然类的正则化学习算法与近因偏差的形式实现更快的收敛速度,以近似的效率和粗糙的相关均衡多人范式游戏。当游戏中的每个玩家使用我们班的算法时,他们的个人遗憾以$O(T^{-3/4})$衰减,而效用之和则以$O(T^{-1})$收敛到近似最佳值--这是对最坏情况下$O(T^{-1/2})$率的改进。我们展示了一个黑盒减少类中的任何算法,以实现对对手的$\tilde{O}(T^{-1/2})$率,同时保持对类中的算法更快的速度。我们的结果扩展了[Rakhlin and Shridharan 2013]和[Daskalakis et al. 2014]的结果,他们只分析了特定算法的两人零和游戏。