NOCAP：近最优相关感知分区连接
NOCAP: Near-Optimal Correlation-Aware Partitioning Joins
AI总结：
针对混合哈希连接无法充分利用连接属性相关分布、存在相关倾斜时分区次优的问题，提出内存可调的近最优相关感知分区连接算法NOCAP，在多种内存预算和相关分布场景下性能均优于现有算法。
摘要：
由于内存预算并非总能随数据规模扩展，基于存储的连接算法如今仍被广泛使用。在已开发的众多连接算法中，混合哈希连接（Hybrid Hash Join，HHJ）已被广泛部署且被证明高效，其设计目标是利用所有可用内存，最大化直接在内存中完成连接的数据量。然而，HHJ无法充分利用连接属性相关分布的详细信息。\n本文表明，当连接属性存在相关倾斜时，HHJ的分区方式并非最优。为此，我们针对最常见的连接类型之一——主键-外键（PK-FK）连接，通过一种新的基于代价的分区连接分析推导得出了最优分区策略。该最优分区策略内存代价较高，因此我们进一步推导了一种近似算法，其内存代价可调，且能取得近最优结果。我们的算法名为NOCAP（Near-Optimal Correlation-Aware Partitioning，近最优相关感知分区）连接，在存在相关倾斜的场景下，性能较当前最优算法最高提升30%，较经典的Grace Hash Join最高提升4倍。此外，在内存预算有限的情况下，即使相关分布均匀，NOCAP的性能也较HHJ最高提升10%。总体而言，在内存预算从低于√||关系||到高于||关系||的变化范围内，NOCAP的表现均优于现有最优算法，且性能始终与对应内存预算下的最佳算法相当。
英文摘要：
Storage-based joins are still commonly used today because the memory budget does not always scale with the data size. One of the many join algorithms developed that has been widely deployed and proven to be efficient is the Hybrid Hash Join (HHJ), which is designed to exploit any available memory to maximize the data that is joined directly in memory. However, HHJ cannot fully exploit detailed knowledge of the join attribute correlation distribution. In this paper, we show that given a correlation skew in the join attributes, HHJ partitions data in a suboptimal way. To do that, we derive the optimal partitioning using a new cost-based analysis of partitioning-based joins that is tailored for primary key - foreign key (PK-FK) joins, one of the most common join types. This optimal partitioning strategy has a high memory cost, thus, we further derive an approximate algorithm that has tunable memory cost and leads to near-optimal results. Our algorithm, termed NOCAP (Near-Optimal Correlation-Aware Partitioning) join, outperforms the state-of-the-art for skewed correlations by up to $30\%$, and the textbook Grace Hash Join by up to $4\times$. Further, for a limited memory budget, NOCAP outperforms HHJ by up to $10\%$, even for uniform correlation. Overall, NOCAP dominates state-of-the-art algorithms and mimics the best algorithm for a memory budget varying from below $\sqrt{\|\text{relation}\|}$ to more than $\|\text{relation}\|$.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
