定制OLAP:合成工作负载特定的定制数据库引擎
Bespoke OLAP: Synthesizing Workload-Specific One-size-fits-one Database Engines
AI总结:
Bespoke OLAP通过自动化合成流程,生成针对特定工作负载的高性能数据库引擎,实现比通用系统高多个数量级的性能提升。
摘要:
现代OLAP引擎设计用于支持任意分析工作负载,但这种通用性带来了结构开销,包括运行时模式解释、间接层和抽象边界,即使在高度优化的系统中也是如此。一个针对固定工作负载专门设计的引擎可以消除这些成本,并利用工作负载特定的数据结构和执行算法,从而实现显著更高的性能。历史上,构建此类定制引擎在经济上并不实际,因为手动工程工作量很大。最近基于大语言模型(LLM)的代码合成技术挑战了这种权衡,通过使系统生成自动化。然而,简单地提示LLM生成数据库引擎并不能产生正确或高效的架构,因为有效的合成需要系统的性能反馈、结构化的细化和对深度架构依赖关系的仔细管理。我们提出了Bespoke OLAP,一种完全自动化的合成流程,用于构建紧密针对给定工作负载的高性能数据库引擎。我们的方法整合了迭代性能评估和自动化验证,以指导从存储到查询执行的合成过程。我们证明Bespoke OLAP可以在几分钟到几小时内从头生成一个针对特定工作负载的引擎,其性能比现代通用系统如DuckDB高出数量级。
英文摘要:
Modern OLAP engines are designed to support arbitrary analytical workloads, but this generality incurs structural overhead, including runtime schema interpretation, indirection layers, and abstraction boundaries, even in highly optimized systems. An engine specialized to a fixed workload can eliminate these costs and exploit workload-specific data structures and execution algorithms for substantially higher performance. Historically, constructing such bespoke engines has been economically impractical due to the high manual engineering effort. Recent advances in LLM-based code synthesis challenge this tradeoff by enabling automated system generation. However, naively prompting an LLM to produce a database engine does not yield a correct or efficient design, as effective synthesis requires systematic performance feedback, structured refinement, and careful management of deep architectural interdependencies. We present Bespoke OLAP, a fully autonomous synthesis pipeline for constructing high-performance database engines tightly tailored to a given workload. Our approach integrates iterative performance evaluation and automated validation to guide synthesis from storage to query execution. We demonstrate that Bespoke OLAP can generate a workload-specific engine from scratch within minutes to hours, achieving order-of-magnitude speedups over modern general-purpose systems such as DuckDB.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。