arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2003.09541cs.DB
收藏

用于交互式极端规模分析的Synopses数据引擎

A Synopses Data Engine for Interactive Extreme-Scale Analytics

Antonis Kontaxakis, Nikos Giatrakos, Antonios Deligiannakis

摘要:

在这项工作中,我们详细介绍了一个Synopses数据引擎的设计和结构,它结合了并行处理和流摘要的优点,以提供交互式分析在极端的规模。我们的服务器构建在Apache Flink之上,并实现了一个概要即服务的范例。因为它实现了(a)按需为数千个流同时维护数千个各种类型的概要,(b)在各种并发工作流之间重用维护的概要,(c)甚至为跨(大数据)平台工作流提供数据摘要设施,(d)新概要的可插入性,(e)增加了工作流执行优化的潜力。所提出的方法对于极端规模的交互式分析是有用的,因为它实现了(i)增强的水平可扩展性,即,不仅将计算扩展到计算机集群中可用的多个处理单元,而且通过对精心制作的数据摘要进行操作来利用分配给每个处理单元的处理负载,(i i)垂直可扩展性,即,将计算缩放到非常高数量的处理流,以及(iii)联合可缩放性即,通过控制回答在多个潜在地理分散的集群上提出的全局查询所需的通信,将计算扩展到单个集群和云之外。

英文摘要:

In this work, we detail the design and structure of a Synopses Data Engine (SDE) which combines the virtues of parallel processing and stream summarization towards delivering interactive analytics at extreme scale. Our SDE is built on top of Apache Flink and implements a synopsis-as-a-service paradigm. In that it achieves (a) concurrently maintaining thousands of synopses of various types for thousands of streams on demand, (b) reusing maintained synopses among various concurrent workflows, (c) providing data summarization facilities even for cross-(Big Data) platform workflows, (d) pluggability of new synopses on-the-fly, (e) increased potential for workflow execution optimization. The proposed SDE is useful for interactive analytics at extreme scales because it enables (i) enhanced horizontal scalability, i.e., not only scaling out the computation to a number of processing units available in a computer cluster, but also harnessing the processing load assigned to each by operating on carefully-crafted data summaries, (ii) vertical scalability, i.e., scaling the computation to very high numbers of processed streams and (iii) federated scalability i.e., scaling the computation beyond single clusters and clouds by controlling the communication required to answer global queries posed over a number of potentially geo-dispersed clusters.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑