Modularis：异构分布式平台上的模块化关系分析
Modularis: Modular Relational Analytics over Heterogeneous Distributed Platforms
摘要：
每天产生的大量数据以及数据分析的进步导致了数据管理和分析系统的激增。通常，这些系统是围绕针对底层硬件优化的高度专业化的单片运算符构建的。虽然在短期内有效，但这种方法使运营商难以移植和适应，由于算法和硬件发展的速度，这一要求越来越高。为了解决这一限制，我们提出了Modularis，一个基于子运算符的数据分析执行层，即，类似于传统数据库操作符但粒度更细的可组合构建块。为了证明我们的方法的优势，我们使用Modularis构建一个分布式查询处理系统，支持在RDMA集群，无服务器云平台和智能存储引擎上运行的关系查询。Modularis需要最少的代码更改来跨这三个不同的硬件平台执行查询，这表明子运算符方法减少了代码的数量和复杂性。事实上，平台的变化只影响依赖底层硬件的子运营商。我们通过将Modularis与SQL处理框架（Presto），商业集群数据库（SingleStore）以及查询即服务系统（Athena，BigQuery）进行比较，展示了Modularis的端到端性能。Modularis优于所有这些系统，证明了模块化设计的设计和架构优势可以在不降低性能的情况下实现。我们还比较了Modularis与手工优化的RDMA集群连接实现。我们表明，Modularis的优点是可以很容易地扩展到更广泛的连接变体和组查询，所有这些都不支持手工调优连接。
英文摘要：
The enormous quantity of data produced every day together with advances in data analytics has led to a proliferation of data management and analysis systems. Typically, these systems are built around highly specialized monolithic operators optimized for the underlying hardware. While effective in the short term, such an approach makes the operators cumbersome to port and adapt, which is increasingly required due to the speed at which algorithms and hardware evolve. To address this limitation, we present Modularis, an execution layer for data analytics based on sub-operators, i.e.,composable building blocks resembling traditional database operators but at a finer granularity. To demonstrate the advantages of our approach, we use Modularis to build a distributed query processing system supporting relational queries running on an RDMA cluster, a serverless cloud platform, and a smart storage engine. Modularis requires minimal code changes to execute queries across these three diverse hardware platforms, showing that the sub-operator approach reduces the amount and complexity of the code. In fact, changes in the platform affect only sub-operators that depend on the underlying hardware. We show the end-to-end performance of Modularis by comparing it with a framework for SQL processing (Presto), a commercial cluster database (SingleStore), as well as Query-as-a-Service systems (Athena, BigQuery). Modularis outperforms all these systems, proving that the design and architectural advantages of a modular design can be achieved without degrading performance. We also compare Modularis with a hand-optimized implementation of a join for RDMA clusters. We show that Modularis has the advantage of being easily extensible to a wider range of join variants and group by queries, all of which are not supported in the hand-tuned join.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
