arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2211.09100cs.LG
收藏

基于参数化函数逼近的全局优化

Global Optimization with Parametric Function Approximation

University of California, Santa Barbara(加州大学圣巴巴拉分校)

Chong Liu, Yu-Xiang Wang

AI总结:

针对带噪声零阶全局优化的维度灾难问题,提出基于参数化函数族的GO-UCB算法,通过梯度构造参数不确定性集实现乐观探索,理论上达到Õ(√T)累积遗憾,实验效果优于主流贝叶斯优化方法。

摘要:

我们研究带噪声零阶预言机的全局优化问题——这是一个具有实际意义的问题,可应用于从深度学习超参数调优到新材料设计等多种场景。现有方法依赖高斯过程或其他非参数族,受困于维度灾难。本文提出一种新算法GO-UCB,转而利用参数化函数族(例如神经网络)。在可实现假设和少量其他温和几何条件下,我们证明GO-UCB的累积遗憾为Õ(√T),其中T为时间步长。GO-UCB的核心是基于梯度精心设计的参数不确定性集合,可实现乐观探索。仿真与真实实验表明,即使模型存在误设,GO-UCB的表现也优于主流贝叶斯优化方法。

英文摘要:

We consider the problem of global optimization with noisy zeroth order oracles - a well-motivated problem useful for various applications ranging from hyper-parameter tuning for deep learning to new material design. Existing work relies on Gaussian processes or other non-parametric family, which suffers from the curse of dimensionality. In this paper, we propose a new algorithm GO-UCB that leverages a parametric family of functions (e.g., neural networks) instead. Under a realizable assumption and a few other mild geometric conditions, we show that GO-UCB achieves a cumulative regret of Õ$(\sqrt{T})$ where $T$ is the time horizon. At the core of GO-UCB is a carefully designed uncertainty set over parameters based on gradients that allows optimistic exploration. Synthetic and real-world experiments illustrate GO-UCB works better than popular Bayesian optimization approaches, even if the model is misspecified.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑