Lynceus：经济高效的数据分析作业调优和配置
Lynceus: Cost-efficient Tuning and Provisioning of Data Analytic Jobs
摘要：
现代数据分析和机器学习工作在云中找到了一个自然的部署平台，以满足其众所周知的大量资源需求。然而，为了实现成本效率，识别满足用户定义的QoS约束（例如，在执行时间上），同时避免不必要的过度供应。本文介绍了Lynceus，一种用于优化基于云的数据分析工作的新方法，该方法通过在最终推荐配置和用于推荐配置的优化过程方面实现显着的成本节约来改进最先进的方法。与现有解决方案不同，Lynceus以联合方式优化云相关和应用程序级参数。这使得推荐配置的成本在第90百分位数时比现有方法降低了3.7倍，现有方法将云相关参数和应用程序级参数的优化视为两个独立的问题。此外，Lynceus降低了优化过程的成本（即，测试配置所产生的云成本）高达11倍。i）超时方法，其允许中止被认为是次优的配置的探索，同时仍然提取有用的信息以指导未来的探索并改进其预测模型-与最近的工作不同，其要么招致测试次优配置的全部成本，要么不能从中止的运行中提取任何知识; ii）一种具有远见和洞察力的技术，通过预测每次探索的长期影响来确定要测试哪些配置-这与采用贪婪优化方法的最先进的云作业优化方法不同。
英文摘要：
Modern data analytic and machine learning jobs find in the cloud a natural deployment platform to satisfy their notoriously large resource requirements. Yet, to achieve cost efficiency, it is crucial to identify a deployment configuration that satisfies user-defined QoS constraints (e.g., on execution time), while avoiding unnecessary over-provisioning. This paper introduces Lynceus, a new approach for the optimization of cloud based data analytic jobs that improves overstate-of-the-art approaches by enabling significant cost savings both in terms of the final recommended configuration and of the optimization process used to recommend configurations. Unlike existing solutions, Lynceus optimizes in a joint fashion both the cloud-related and the application-level parameters. This allows for a reduction of the cost of recommended configurations by up to 3.7x at the 90-th percentile with respect to existing approaches, which treat the optimization of cloud-related and application-level parameters as two independent problems. Further, Lynceus reduces the cost of the optimization process (i.e., the cloud cost incurred for testing configurations) by up to 11x. Such an improvement is achieved thanks to two mechanisms: i) a timeout approach which allows to abort the exploration of configurations that are deemed suboptimal, while still extracting useful information to guide future explorations and to improve its predictive model - differently from recent works, which either incur the full cost for testing suboptimal configurations or are unable to extract any knowledge from aborted runs; ii) a long-sighted and budget-aware technique that determines which configurations to test by predicting the long-term impact of each exploration - unlike state-of-the-art approaches for the optimization of cloud jobs, which adopt greedy optimization methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
