arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2205.07525cs.LGmath.OCstat.ML
收藏

高维大规模优化问题的一种模型聚合方法

A model aggregation approach for high-dimensional large-scale optimization

National University of Singapore(新加坡国立大学);Arizona State University(亚利桑那州立大学)

Haowei Wang, Ercong Zhang, Szu Hui Ng, Giulia Pedrielli

摘要:

贝叶斯优化在机器学习和仿真优化中有着广泛的应用。随着这些领域计算资源和存储容量的增加,高维和大规模问题变得越来越普遍。在这项研究中,我们提出了一个模型聚合方法在贝叶斯优化(Mambo)算法有效地解决高维大规模优化问题。Mambo使用子采样和子空间嵌入的组合来共同解决高维和大规模问题;此外,还采用模型聚合方法来解决应用嵌入时出现的代理模型不确定性问题。这种代理模型不确定性问题在嵌入文献和实践中很大程度上被忽视,并且当问题是高维的并且数据有限时,它会加剧。我们提出的模型聚合方法降低了这些低维代理模型的风险,提高了BO算法的鲁棒性。我们推导出一个渐近界建议的聚合代理模型,并证明了收敛的Mambo。基准数值实验表明,我们的算法实现了优越的或类似的性能,其他常用的高维BO算法。此外,我们将Mambo应用到用于人脸检测的机器学习算法的级联分类器中,结果表明,Mambo发现设置比基准设置实现更高的分类精度,并且计算速度比其他高维BO算法更快。

英文摘要:

Bayesian optimization (BO) has been widely used in machine learning and simulation optimization. With the increase in computational resources and storage capacities in these fields, high-dimensional and large-scale problems are becoming increasingly common. In this study, we propose a model aggregation method in the Bayesian optimization (MamBO) algorithm for efficiently solving high-dimensional large-scale optimization problems. MamBO uses a combination of subsampling and subspace embeddings to collectively address high dimensionality and large-scale issues; in addition, a model aggregation method is employed to address the surrogate model uncertainty issue that arises when embedding is applied. This surrogate model uncertainty issue is largely ignored in the embedding literature and practice, and it is exacerbated when the problem is high-dimensional and data are limited. Our proposed model aggregation method reduces these lower-dimensional surrogate model risks and improves the robustness of the BO algorithm. We derive an asymptotic bound for the proposed aggregated surrogate model and prove the convergence of MamBO. Benchmark numerical experiments indicate that our algorithm achieves superior or comparable performance to other commonly used high-dimensional BO algorithms. Moreover, we apply MamBO to a cascade classifier of a machine learning algorithm for face detection, and the results reveal that MamBO finds settings that achieve higher classification accuracy than the benchmark settings and is computationally faster than other high-dimensional BO algorithms.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑