arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.01027cs.DBcs.LG
收藏

PRICE:一种用于跨数据库基数估计的预训练模型

PRICE: A Pretrained Model for Cross-Database Cardinality Estimation

Alibaba Group(阿里巴巴集团);University of Science and Technology of China(中国科学技术大学);Renmin University of China(中国人民大学)

Tianjing Zeng, Junwei Lan, Jiahong Ma, Wenqing Wei, Rong Zhu, Pengfei Li, Bolin Ding, Defu Lian, Zhewei Wei, Jingren Zhou

AI总结:

本文提出预训练多表基数估计模型PRICE,利用可迁移特征和自注意力学习元知识,以极低准备成本在未见数据库上实现高准确率,并可通过微调接近最优执行计划。

摘要:

基数估计(CardEst)对于优化查询执行计划至关重要。近年来基于机器学习的CardEst方法虽然取得了较高的准确率,但由于准备成本高且缺乏跨数据库的可迁移性,在实际部署中面临挑战。本文提出了PRICE,一种预训练的多表CardEst模型,以解决这些局限性。PRICE采用与数据分布和查询信息相关的低层次但可迁移的特征,并巧妙地应用自注意力模型来学习元知识,从而在任意数据库中计算基数。它普遍适用于任何未见过的全新数据库,能够获得较高的估计准确率,而其准备成本仅与基本的一维直方图CardEst方法相当。此外,PRICE还可以通过微调进一步提升其在特定数据库上的性能。我们使用30个多样化的数据集对PRICE进行了预训练,整个过程约在5小时内完成,最终模型大小仅为约40MB。评估结果表明,PRICE始终优于现有方法,在多个未见过的数据库上取得了最高的估计准确率,并以更低的开销生成更快的执行计划。在使用少量数据库特定查询进行微调后,PRICE甚至能够找到非常接近最优的执行计划。同时,PRICE普遍适用于数据更新、数据扩展和查询负载变化等不同场景。我们已在 https://github.com/StCarmen/PRICE 公开了所有数据和代码。

英文摘要:

Cardinality estimation (CardEst) is essential for optimizing query execution plans. Recent ML-based CardEst methods achieve high accuracy but face deployment challenges due to high preparation costs and lack of transferability across databases. In this paper, we propose PRICE, a PRetrained multI-table CardEst model, which addresses these limitations. PRICE takes low-level but transferable features w.r.t. data distributions and query information and elegantly applies self-attention models to learn meta-knowledge to compute cardinality in any database. It is generally applicable to any unseen new database to attain high estimation accuracy, while its preparation cost is as little as the basic one-dimensional histogram-based CardEst methods. Moreover, PRICE can be finetuned to further enhance its performance on any specific database. We pretrained PRICE using 30 diverse datasets, completing the process in about 5 hours with a resulting model size of only about 40MB. Evaluations show that PRICE consistently outperforms existing methods, achieving the highest estimation accuracy on several unseen databases and generating faster execution plans with lower overhead. After finetuning with a small volume of databasespecific queries, PRICE could even find plans very close to the optimal ones. Meanwhile, PRICE is generally applicable to different settings such as data updates, data scaling, and query workload shifts. We have made all of our data and codes publicly available at https://github.com/StCarmen/PRICE.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑