使用廉价、瞬时云资源的弹性临时数据存储
An Elastic Ephemeral Datastore using Cheap, Transient Cloud Resources
摘要:
Spot实例是以60-90%的成本提供的虚拟机,可以随时回收,只有很短的警告期。Spot实例已经被用于显著降低云中处理工作负载的成本。然而,利用spot实例来降低有状态云应用程序的成本更具挑战性,因为突然的抢占会导致数据丢失。在这项工作中,我们建议利用现货实例,以减少分布式数据分析应用程序中的短暂数据管理的成本。我们专门针对短暂数据,因为现代分析工作负载中的这一大类数据具有较低的持久性要求;如果丢失,可以通过重新执行计算任务来重新生成数据。我们设计了一个弹性的,分布式的短暂的数据交换,处理节点抢占透明的用户应用程序和最大限度地减少数据丢失,在节点抢占警告期间重新分配数据。我们在ApacheCrailTM上实现了我们的弹性服务器,并使用各种工作负载和VM类型来评估系统。通过利用现货实例,我们表明,我们可以运行TPC-DS查询,与使用按需虚拟机的TPC-DS查询相比,成本降低了60%,而端到端的执行时间只增加了2.1%。
英文摘要:
Spot instances are virtual machines offered at 60-90% lower cost that can be reclaimed at any time, with only a short warning period. Spot instances have already been used to significantly reduce the cost of processing workloads in the cloud. However, leveraging spot instances to reduce the cost of stateful cloud applications is much more challenging, as the sudden preemptions lead to data loss. In this work, we propose leveraging spot instances to decrease the cost of ephemeral data management in distributed data analytics applications. We specifically target ephemeral data as this large class of data in modern analytics workloads has low durability requirements; if lost, the data can be regenerated by re-executing compute tasks. We design an elastic, distributed ephemeral datastore that handles node preemptions transparently to user applications and minimizes data loss by redistributing data during node preemption warning periods. We implement our elastic datastore on top of the Apache Crail datastore and evaluate the system with various workloads and VM types. By leveraging spot instances, we show that we can run TPC-DS queries with 60\% lower cost compared to using on-demand VMs for the datastore, while only increasing end-to-end execution time by 2.1%.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。