基于轨迹数据的性能回归方法
Towards Feature-Based Performance Regression Using Trajectory Data
摘要：
黑盒优化是一个非常活跃的研究领域，每年都有许多新的算法被开发出来。这种多样性是必要的，一方面，因为不同的算法最适合不同类型的优化问题。但这种多样性也带来了一个元问题：针对手头的给定问题选择哪种算法？过去的研究表明，基于探索性景观分析（ELA）的每实例算法选择可以是一种有效的手段来解决这个元问题。然而，现有的方法需要基于大量样本的问题特征的近似，这些样本通常通过均匀采样或拉丁超立方体设计来选择。对这些点的评估是昂贵的，因此基于ELA的算法选择相对于默认算法的益处必须是显著的，以便获得回报。人们可能希望通过使用默认算法无论如何都会执行的样本来绕过对特征近似的评估，即，通过使用默认算法的轨迹点。我们在本文中分析如何以及这样的方法可以工作。具体来说，我们测试如何准确地基于ELA方法可以预测的CMA-ES的最终解决方案的质量后，一个固定的预算功能评估。我们观察到，基于概率的预测的损失可以是令人惊讶的小相比，经典的全球抽样方法，如果剩余的预算，解决方案的质量预测是不是太大。相比之下，特征选择在我们的实验中没有显示出任何优势，而是导致预测准确性恶化。CMA-ES模型中状态变量的引入对预测精度的影响不大。
英文摘要：
Black-box optimization is a very active area of research, with many new algorithms being developed every year. This variety is needed, on the one hand, since different algorithms are most suitable for different types of optimization problems. But the variety also poses a meta-problem: which algorithm to choose for a given problem at hand? Past research has shown that per-instance algorithm selection based on exploratory landscape analysis (ELA) can be an efficient mean to tackle this meta-problem. Existing approaches, however, require the approximation of problem features based on a significant number of samples, which are typically selected through uniform sampling or Latin Hypercube Designs. The evaluation of these points is costly, and the benefit of an ELA-based algorithm selection over a default algorithm must therefore be significant in order to pay off. One could hope to by-pass the evaluations for the feature approximations by using the samples that a default algorithm would anyway perform, i.e., by using the points of the default algorithm's trajectory. We analyze in this paper how well such an approach can work. Concretely, we test how accurately trajectory-based ELA approaches can predict the final solution quality of the CMA-ES after a fixed budget of function evaluations. We observe that the loss of trajectory-based predictions can be surprisingly small compared to the classical global sampling approach, if the remaining budget for which solution quality shall be predicted is not too large. Feature selection, in contrast, did not show any advantage in our experiments and rather led to worsened prediction accuracy. The inclusion of state variables of CMA-ES only has a moderate effect on the prediction accuracy.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
