Tuneful：用于大数据分析的在线重要性感知配置调优器
Tuneful: An Online Significance-Aware Configuration Tuner for Big Data Analytics
摘要：
Spark等分布式分析引擎是处理超大型数据集的常见选择。然而，为这些系统找到良好的配置仍然具有挑战性，每个工作负载可能需要不同的设置才能最佳运行。使用次优配置会导致显著的额外运行时成本。此外，Spark和类似的平台在数据科学家社区中越来越受欢迎，这些社区对这些问题的认识相对较低。我们提出了Tuneful，一种有效地调整内存中的集群计算系统的配置的方法。Tuneful结合了增量灵敏度分析和贝叶斯优化，使用少量的执行从高维搜索空间中识别接近最优的配置。这种设置允许在线进行调整，而无需任何先前的培训。我们的实验结果表明，Tuneful减少了搜索时间找到接近最佳配置的62%（中位数）相比，现有的国家的最先进的技术。这意味着调优成本的摊销发生得更快，从而能够针对新的工作负载类别进行实际调优。
英文摘要：
Distributed analytics engines such as Spark are a common choice for processing extremely large datasets. However, finding good configurations for these systems remains challenging, with each workload potentially requiring a different setup to run optimally. Using suboptimal configurations incurs significant extra runtime costs. %Furthermore, Spark and similar platforms are gaining traction within data-scientists communities where awareness of such issues is relatively low. We propose Tuneful, an approach that efficiently tunes the configuration of in-memory cluster computing systems. Tuneful combines incremental Sensitivity Analysis and Bayesian optimization to identify near-optimal configurations from a high-dimensional search space, using a small number of executions. This setup allows the tuning to be done online, without any previous training. Our experimental results show that Tuneful reduces the search time for finding close-to-optimal configurations by 62\% (at the median) when compared to existing state-of-the-art techniques. This means that the amortization of the tuning cost happens significantly faster, enabling practical tuning for new classes of workloads.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
