JetStream:为现有数据库系统生成查询加速器
JetStream: Generating Query Accelerators for Existing Database Systems
AI总结:
JetStream是扩展现有DBMS的查询专用加速器生成系统,经TPC-H等测试,其生成的有状态加速器对DuckDB等实现数百倍加速,还可泛化到新工作负载,能支持动态工作负载。
AI中文摘要:
近期研究表明,大语言模型(LLMs)可针对固定工作负载合成高度专用的数据库系统,但现有方法通常假设数据是静态的,且会用生成的表示替换数据库的原生存储。本文提出JetStream,这是一种生成查询专用加速器的系统,它会扩展现有数据库管理系统(DBMS)而非替换它。JetStream包含三个部分:第一,分阶段、基于测量的智能体工作流会生成并优化查询专用加速器,包括在有益时生成持久辅助状态;第二,固定的引擎中立底层会提供执行、事务协调、状态管理及维护的通用接口;第三,单独的合成工作流会生成特定于引擎的后端适配器,用于连接底层与底层引擎。对于有状态加速器,JetStream还会生成维护逻辑,并使用运行时策略在数据库发生变化时选择增量维护、重建和懒修复。在SF=20的TPC-H基准测试中,JetStream生成的有状态加速器相比DuckDB实现了833倍的几何平均只读加速,而GenDB为34.07倍,Bespoke OLAP为12.35倍;在每60秒执行一次TPC-H刷新的场景下,JetStream仍保持了375倍的几何平均工作负载加速。此外,JetStream可泛化到新的未见过的工作负载,相比DuckDB实现了102倍的几何平均只读加速,相比PostgreSQL实现了486倍的几何平均只读加速。这些结果表明,高度激进的生成式专用化可与现有DBMS无缝集成,并支持动态工作负载。
英文摘要:
Recent work has shown that LLMs can synthesize highly specialized database systems for fixed workloads, but existing approaches typically assume static data and replace the database's native storage with generated representations. We present JetStream, a system for generating query-specific accelerators that instead extend an existing DBMS. JetStream consists of three parts. First, a staged, measurement-driven agentic workflow generates and optimizes query-specific accelerators, including persistent auxiliary state when beneficial. Second, a fixed, engine-neutral substrate provides the common interfaces for execution, transaction coordination, state management, and maintenance. Third, a separate synthesis workflow generates engine-specific backend adapters that connect the substrate to the underlying engine. For stateful accelerators, JetStream also generates maintenance logic and uses a runtime policy to choose among incremental maintenance, rebuilds, and lazy repair as the database changes. On TPC-H at SF=20, stateful accelerators generated by JetStream achieve an 833x geomean read-only speedup over DuckDB, compared with 34.07x for GenDB and 12.35x for Bespoke OLAP. Under TPC-H refreshes every 60 seconds, JetStream maintains a 375x geomean workload speedup. We also show that JetStream generalizes to new, unseen workloads, achieving geomean read-only speedups of 102x over DuckDB and 486x over PostgreSQL. These results show that aggressive generated specialization can be integrated seamlessly with existing DBMSes and support dynamic workloads.