FLAT：快速、轻量、精确的基数估计方法
FLAT: Fast, Lightweight and Accurate Method for Cardinality Estimation
摘要：
查询优化器依赖于准确的基数估计（CardEst）来生成良好的执行计划。CardEst的核心问题是如何以准确和紧凑的方式对丰富的属性联合分布建模。尽管已有几十年的研究，现有的方法要么只使用独立因子分解来简化模型，导致估计不准确，要么通过无损条件因子分解而没有任何独立假设，导致概率计算缓慢。在本文中，我们提出了FLAT，CardEst方法，同时是快速的概率计算，轻量级的模型大小和准确的估计质量。FLAT的核心思想是一种新的无监督图形模型，称为FSPN。该方法利用独立因子分解和条件因子分解对不同层次的属性相关性进行自适应建模，从而综合了两者的优点。FLAT支持在底层FSPN模型上以接近线性时间进行有效的在线概率计算，提供有效的离线模型构建，并支持增量模型更新。它可以估计单表查询和多表连接查询的基数。大量的实验研究表明，FLAT优于现有的CardEst方法在著名的IMDB基准测试：FLAT实现1至5个数量级更好的准确性，1至3个数量级更快的概率计算速度和1至2个数量级更低的存储成本。我们还将FLAT集成到Postgres中以执行端到端测试。在基准测试工作负载上，它将查询执行时间提高了12.9%，非常接近使用真实基数的最佳结果14.2%。
英文摘要：
Query optimizers rely on accurate cardinality estimation (CardEst) to produce good execution plans. The core problem of CardEst is how to model the rich joint distribution of attributes in an accurate and compact manner. Despite decades of research, existing methods either over simplify the models only using independent factorization which leads to inaccurate estimates, or over complicate them by lossless conditional factorization without any independent assumption which results in slow probability computation. In this paper, we propose FLAT, a CardEst method that is simultaneously fast in probability computation, lightweight in model size and accurate in estimation quality. The key idea of FLAT is a novel unsupervised graphical model, called FSPN. It utilizes both independent and conditional factorization to adaptively model different levels of attributes correlations, and thus dovetails their advantages. FLAT supports efficient online probability computation in near liner time on the underlying FSPN model, provides effective offline model construction and enables incremental model updates. It can estimate cardinality for both single table queries and multi table join queries. Extensive experimental study demonstrates the superiority of FLAT over existing CardEst methods on well known IMDB benchmarks: FLAT achieves 1 to 5 orders of magnitude better accuracy, 1 to 3 orders of magnitude faster probability computation speed and 1 to 2 orders of magnitude lower storage cost. We also integrate FLAT into Postgres to perform an end to end test. It improves the query execution time by 12.9% on the benchmark workload, which is very close to the optimal result 14.2% using the true cardinality.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
