发表机构
University of Birmingham; IDEAS Lab, University of Birmingham（伯明翰大学; 伯明翰大学IDEAS实验室）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究在不同预算下可配置系统中优化器的选择问题，通过对22个系统的8种优化器进行评估，发现基于模型的优化器在预算紧张时适用，无模型优化器在预算充足时更优，FLASH在多数系统中无论预算如何都表现良好，并探究了原因。
AI 中文摘要
软件配置调优对优化系统性能至关重要，过去十年出现了各种优化器。调优过程所需时间因系统而异，同一系统中用户预算和偏好设置也不同。本文旨在回答给定预算水平下，哪种优化器是软件工程师的最佳选择这一问题。通过在不同预算水平下对22个可配置系统的8种优化器进行系统评估，发现基于模型的优化器在预算紧张时适用，无模型优化器在预算充足时更优，而FLASH在多数系统中无论预算如何都表现良好。最后探究了原因，许多系统有良好的局部最优，使贪婪优化器能有出色表现。
英文摘要
Software configuration tuning is crucial for optimising system performance, and various optimisers have emerged over the last decade. Yet, the time required during the tuning process may vary across systems. In some systems (e.g., PostgreSQL), it may take a few minutes to measure a configuration, whereas in some others (e.g., MariaDB), it can take several hours. Moreover, even within the same system, users may have varying budgets and preferred settings. This naturally raises a question -- Given a budget level, which optimiser is the best choice for SE practitioners? This matters because optimisers usually have their own ``comfort zone'' and may perform very differently under distinct budgets. In this paper, we aim to answer this question. We systematically evaluate eight well-established optimisers across 22 configurable systems under varying budget levels. We find that, unsurprisingly, model-based optimisers (e.g., SMAC) are well-suited under tight budgets, and model-free optimisers (e.g., GAs) become superior with more generous budgets. However, interestingly, there is one optimiser, FLASH, that performs consistently well on most systems regardless of budgets. We lastly investigate the reasons behind this phenomenon and find that many systems possess good local optima (with large basins of attraction), allowing greedy optimisers (e.g., FLASH) to achieve strong performance. Source code, data, and supplementary materials of this work are available at https://anonymous.4open.science/r/Config-W2W-98B2.
