等值连接的扩展与负载均衡
Scaling and Load-Balancing Equi-Joins
AI总结：
本文提出 AM-Join 及 IB-Join 算法族，通过 Tree-Join 与 Broadcast-Join 自适应协同解决分布式无共享架构中的等值连接倾斜与负载均衡问题。
摘要：
连接两张表的任务是数据库查询的基础。本文聚焦等值连接问题：当来自两张被连接表的一对记录在连接列上的值相等时，该记录对就属于连接结果的一部分。当被连接表中的记录数相对较小时，这是一个易处理的问题；但随着表规模增大，它会变得极具挑战性，尤其是当两张被连接表中都存在热键（即对应大量记录的连接列值）时。本文是 [metwally-SIGMOD-2022] 的扩展版本，提出了 Adaptive-Multistage-Join（AM-Join），用于在分布式无共享架构中实现可扩展且快速的等值连接。AM-Join 利用了：(a) Tree-Join，这是一种所提出的新颖算法，在被连接表共享热键时具有良好的扩展性；以及 (b) Broadcast-Join，这是已知在连接仅在一张表中为热键的键时最快的算法。与最先进算法不同，AM-Join：(a) 通过在整个连接执行过程中实现负载均衡，整体性地解决连接倾斜问题；并且 (b) 支持所有外连接变体，无需记录去重或自定义表分区。为获得最快的 AM-Join 外连接性能，我们针对小表-大表连接提出了 Index-Broadcast-Join（IB-Join）算法族，其中一张表可放入内存，而另一张表可能大出几个数量级。IB-Join 的外连接变体改进了最先进的小表-大表外连接算法。所提出的算法可用于任何无共享架构。我们使用 Spark 实现了一个 MapReduce 版本。评估表明，与最先进算法相比，所提出算法的执行速度显著更快，并且能够扩展到倾斜更严重、规模大出几个数量级的表。
英文摘要：
The task of joining two tables is fundamental for querying databases. In this paper, we focus on the equi-join problem, where a pair of records from the two joined tables are part of the join results if equality holds between their values in the join column(s). While this is a tractable problem when the number of records in the joined tables is relatively small, it becomes very challenging as the table sizes increase, especially if hot keys (join column values with a large number of records) exist in both joined tables. This paper, an extended version of [metwally-SIGMOD-2022], proposes Adaptive-Multistage-Join (AM-Join) for scalable and fast equi-joins in distributed shared-nothing architectures. AM-Join utilizes (a) Tree-Join, a proposed novel algorithm that scales well when the joined tables share hot keys, and (b) Broadcast-Join, the known fastest when joining keys that are hot in only one table. Unlike the state-of-the-art algorithms, AM-Join (a) holistically solves the join-skew problem by achieving load balancing throughout the join execution, and (b) supports all outer-join variants without record deduplication or custom table partitioning. For the fastest AM-Join outer-join performance, we propose the Index-Broadcast-Join (IB-Join) family of algorithms for Small-Large joins, where one table fits in memory and the other can be up to orders of magnitude larger. The outer-join variants of IB-Join improves on the state-of-the-art Small-Large outer-join algorithms. The proposed algorithms can be adopted in any shared-nothing architecture. We implemented a MapReduce version using Spark. Our evaluation shows the proposed algorithms execute significantly faster and scale to more skewed and orders-of-magnitude bigger tables when compared to the state-of-the-art algorithms.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
