Tempura:一个通用的基于成本的增量数据处理优化框架(扩展版)
Tempura: A General Cost Based Optimizer Framework for Incremental Data Processing (Extended Version)
摘要:
增量处理在许多应用中被广泛采用,从增量视图维护、流计算到最近出现的渐进式数据仓库和间歇式查询处理。尽管有许多算法开发了这个主题,他们都不能产生一个增量计划,总是达到最佳性能,因为最佳计划是数据相关的。在本文中,我们开发了一种新的基于成本的优化器框架,称为天妇罗,用于优化增量数据处理。我们提出了一个增量查询规划模型称为TIP的概念的基础上,随时间变化的关系,它可以正式建模增量处理的最一般的形式。我们给出了一个完整的规范天妇罗,它不仅可以统一各种现有的技术,以产生一个最佳的增量计划,但也允许开发人员添加他们的重写规则。我们研究如何探索计划空间,寻找最佳的增量计划。我们进行了彻底的实验评估天妇罗在各种增量处理方案,以显示其有效性和效率。
英文摘要:
Incremental processing is widely-adopted in many applications, ranging from incremental view maintenance, stream computing, to recently emerging progressive data warehouse and intermittent query processing. Despite many algorithms developed on this topic, none of them can produce an incremental plan that always achieves the best performance, since the optimal plan is data dependent. In this paper, we develop a novel cost-based optimizer framework, called Tempura, for optimizing incremental data processing. We propose an incremental query planning model called TIP based on the concept of time-varying relations, which can formally model incremental processing in its most general form. We give a full specification of Tempura, which can not only unify various existing techniques to generate an optimal incremental plan, but also allow the developer to add their rewrite rules. We study how to explore the plan space and search for an optimal incremental plan. We conduct a thorough experimental evaluation of Tempura in various incremental processing scenarios to show its effectiveness and efficiency.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。