线性化 Laplace 在贝叶斯优化中的前景与陷阱
Promises and Pitfalls of the Linearized Laplace in Bayesian Optimization
AI总结：
本文评估线性化 Laplace 近似用于贝叶斯优化代理模型的效果，展示其性能与灵活性，并指出若干陷阱及无界搜索空间下的潜在问题。
摘要：
线性化 Laplace 近似（LLA）已被证明在构建贝叶斯神经网络时既有效又高效。它在理论上具有吸引力，因为它可以被视为一个高斯过程后验：其均值函数由神经网络的最大后验预测函数给出，而协方差函数由经验神经正切核诱导。然而，尽管其有效性已在图像分类等大规模任务中得到研究，它却尚未在贝叶斯优化这类序贯决策问题中得到研究；在贝叶斯优化中，具有简单均值函数和径向基函数等核函数的高斯过程是事实上的代理模型。在本工作中，我们研究 LLA 在贝叶斯优化中的实用性，并强调其强劲性能与灵活性。不过，我们也展示了可能出现的一些陷阱，以及当搜索空间无界时 LLA 的一个潜在问题。
英文摘要：
The linearized-Laplace approximation (LLA) has been shown to be effective and efficient in constructing Bayesian neural networks. It is theoretically compelling since it can be seen as a Gaussian process posterior with the mean function given by the neural network's maximum-a-posteriori predictive function and the covariance function induced by the empirical neural tangent kernel. However, while its efficacy has been studied in large-scale tasks like image classification, it has not been studied in sequential decision-making problems like Bayesian optimization where Gaussian processes -- with simple mean functions and kernels such as the radial basis function -- are the de-facto surrogate models. In this work, we study the usefulness of the LLA in Bayesian optimization and highlight its strong performance and flexibility. However, we also present some pitfalls that might arise and a potential problem with the LLA when the search space is unbounded.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
