主成分分析辅助的高维贝叶斯优化
High Dimensional Bayesian Optimization Assisted by Principal Component Analysis
摘要:
贝叶斯优化(BO)是一种代理辅助的全局优化技术,已成功应用于各种领域,例如,自动化机器学习和设计优化。BO技术建立在所谓的填充准则和高斯过程回归(GPR)上,随着搜索空间的维数增加,BO技术遭受显著的计算复杂性和收敛速度的阻碍。将BO扩展到高维优化问题仍然是一项具有挑战性的任务。在本文中,我们建议通过将BO与主成分分析(PCA)混合来解决BO的可扩展性,从而产生一种新的PCA辅助BO(PCA-BO)算法。具体而言,PCA过程在运行期间从所有评估点学习线性变换,并根据评估点的可变性选择变换空间中的维度。然后,我们构建GPR模型,并在所选择的维度所跨越的空间中的填充准则。我们评估我们的PCA-BO的性能的经验收敛速度和CPU时间的多模态问题从COCO基准框架。实验结果表明,PCA-BO算法能有效地减少高维问题的计算时间,并在具有足够全局结构的问题上保持收敛速度。因此,PCA-BO在收敛速度和计算效率之间提供了令人满意的折衷,从而开辟了新的途径来受益于BO方法在高维数值优化中的强度。
英文摘要:
Bayesian Optimization (BO) is a surrogate-assisted global optimization technique that has been successfully applied in various fields, e.g., automated machine learning and design optimization. Built upon a so-called infill-criterion and Gaussian Process regression (GPR), the BO technique suffers from a substantial computational complexity and hampered convergence rate as the dimension of the search spaces increases. Scaling up BO for high-dimensional optimization problems remains a challenging task. In this paper, we propose to tackle the scalability of BO by hybridizing it with a Principal Component Analysis (PCA), resulting in a novel PCA-assisted BO (PCA-BO) algorithm. Specifically, the PCA procedure learns a linear transformation from all the evaluated points during the run and selects dimensions in the transformed space according to the variability of evaluated points. We then construct the GPR model, and the infill-criterion in the space spanned by the selected dimensions. We assess the performance of our PCA-BO in terms of the empirical convergence rate and CPU time on multi-modal problems from the COCO benchmark framework. The experimental results show that PCA-BO can effectively reduce the CPU time incurred on high-dimensional problems, and maintains the convergence rate on problems with an adequate global structure. PCA-BO therefore provides a satisfactory trade-off between the convergence rate and computational efficiency opening new ways to benefit from the strength of BO approaches in high dimensional numerical optimization.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。