FlowLog:通过增量性实现高效且可扩展的Datalog
FlowLog: Efficient and Extensible Datalog via Incrementality
AI总结:
针对现有Datalog系统在效率与可扩展性间的权衡问题,本文提出FlowLog引擎,通过显式关系IR分离递归控制与规则逻辑,实现高效且可扩展的递归计算,在递归工作负载中性能优于现有系统。
摘要:
基于Datalog的语言正重新流行,成为在程序分析和图处理等领域表达递归计算的强大抽象。然而,现有系统常面临效率与可扩展性的权衡:Souffle等引擎通过特定领域设计实现高效,但缺乏通用灵活性;RecStep等通过在传统数据库上层叠Datalog提供模块化,但难以集成Datalog特定优化。本文提出FlowLog填补这一空白,它是一种新Datalog引擎,通过每条规则的显式关系IR清晰分离递归控制(如半朴素执行)与每条规则的逻辑计划。这一边界使我们在逻辑层保留细粒度、Datalog感知的优化,同时在执行时复用现成数据库原语。在逻辑层(即IR),我们应用成熟SQL优化,如逻辑融合和子计划复用。为应对递归工作负载的高波动性,我们采用稳健优先方法,将结构优化器(避免最坏情况连接)与侧向信息传递(早期过滤)配对。FlowLog构建于成熟的流分析框架Differential Dataflow之上,支持批处理和增量Datalog,并添加名为布尔(或代数)特化的新型递归感知优化。评估显示,FlowLog在广泛递归工作负载中优于最先进的Datalog引擎和现代数据库,在保持简单且可扩展架构的同时实现卓越的可扩展性。
英文摘要:
Datalog-based languages are regaining popularity as a powerful abstraction for expressing recursive computations in domains such as program analysis and graph processing. However, existing systems often face a trade-off between efficiency and extensibility. Engines like Souffle achieve high efficiency through domain-specific designs, but lack general-purpose flexibility. Others, like RecStep, offer modularity by layering Datalog on traditional databases, but struggle to integrate Datalog-specific optimizations. This paper bridges this gap by presenting FlowLog, a new Datalog engine that uses an explicit relational IR per-rule to cleanly separate recursive control (e.g., semi-naive execution) from each rule's logical plan. This boundary lets us retain fine-grained, Datalog-aware optimizations at the logical layer, but also reuse off-the-shelf database primitives at execution. At the logical level (i.e. IR), we apply proven SQL optimizations, such as logic fusion and subplan reuse. To address high volatility in recursive workloads, we adopt a robustness-first approach that pairs a structural optimizer (avoiding worst-case joins) with sideways information passing (early filtering). Built atop Differential Dataflow--a mature framework for streaming analytics--FlowLog supports both batch and incremental Datalog and adds novel recursion-aware optimizations called Boolean (or algebraic) specialization. Our evaluation shows that FlowLog outperforms state-of-the-art Datalog engines and modern databases across a broad range of recursive workloads, achieving superior scalability while preserving a simple and extensible architecture.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。