arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2106.09215stat.MLcs.LG
收藏

面向通用高效黑盒优化的最优-统计协同

Optimum-statistical Collaboration Towards General and Efficient Black-box Optimization

Purdue University(普渡大学);University of California Los Angeles(加利福尼亚大学洛杉矶分校)

Wenjie Li, Chi-Hua Wang, Guang Cheng, Qifan Song

AI总结:

本文提出最优-统计协同框架,统一管理黑盒优化中优化误差与统计误差的交互,并据此设计方差自适应算法VHCT,在理论和实验上均取得优于已有方法的通用高效性能。

摘要:

本文对分层赌博机黑盒优化算法中分辨率与统计不确定性的作用做出了关键划分,为更通用的分析和更高效的算法设计提供了指导。我们引入了“最优-统计协同”这一算法框架,用于管理优化过程中优化误差流与统计误差流之间的相互作用。我们在不指定统计误差和不确定性量化器具体形式的情况下,对该框架进行了通用分析。由于该框架及其分析具有通用性,它可以应用于一大类满足不同局部光滑性假设、且具有不同局部最优解数量的函数和划分,这比先前工作所研究的函数类要丰富得多。该框架还启发我们提出了一种更好的统计不确定性度量方法,并由此设计出方差自适应算法 VHCT。在理论上,我们证明了该算法在不同局部光滑性假设下具有速率最优的遗憾界;在实验中,我们展示了该算法在不同设置下优于此前的各种方法。

英文摘要:

In this paper, we make the key delineation on the roles of resolution and statistical uncertainty in hierarchical bandits-based black-box optimization algorithms, guiding a more general analysis and a more efficient algorithm design. We introduce the \textit{optimum-statistical collaboration}, an algorithm framework of managing the interaction between optimization error flux and statistical error flux evolving in the optimization process. We provide a general analysis of this framework without specifying the forms of statistical error and uncertainty quantifier. Our framework and its analysis, due to their generality, can be applied to a large family of functions and partitions that satisfy different local smoothness assumptions and have different numbers of local optimums, which is much richer than the class of functions studied in prior works. Our framework also inspires us to propose a better measure of the statistical uncertainty and consequently a variance-adaptive algorithm \texttt{VHCT}. In theory, we prove the algorithm enjoys rate-optimal regret bounds under different local smoothness assumptions; in experiments, we show the algorithm outperforms prior efforts in different settings.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑