arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1703.07342cs.DB
收藏

LaraDB:一个用于线性和关系代数计算的极小核

LaraDB: A Minimalist Kernel for Linear and Relational Algebra Computation

Dylan Hutchison, Bill Howe, Dan Suciu

摘要:

分析任务使用关系代数(RA)的变体操纵结构化数据,使用线性代数(LA)的变体操纵定量数据。这两个计算模型具有重叠的表达性,从而激发了提供统一推理和算法设计的公共编程模型。在逻辑层面上,我们提出了Lara,一个精益代数的三个运营商,表示RA和LA以及相关的优化规则。我们展示了一系列的证据,位置劳拉%正式和非正式的表达能力的中间件代数的正确水平:更明确的比MapReduce,但比RA或LA更一般。在物理层面上,我们发现,Lara操作符提供了有效的实现,使用一个单一的原语,可在各种后端引擎:范围扫描分区排序的地图。为了评估这些想法,我们在Apache Accumulo中实现了Lara运算符作为范围迭代器,这是Google的BigTable的流行实现。首先,我们展示了劳拉如何表达一个传感器质量控制任务,我们衡量的性能影响的优化劳拉承认这项任务。其次,我们表明LaraDB实现在涉及矩阵乘法形式的连接和聚合的核心任务上优于Accumulo的原生MapReduce集成,特别是在通常不适合横向扩展方法的较小尺度上。我们发现LaraDB提供了一个概念上的精益框架,用于优化混合抽象分析任务,而不会放弃快速的记录级更新和扫描。

英文摘要:

Analytics tasks manipulate structured data with variants of relational algebra (RA) and quantitative data with variants of linear algebra (LA). The two computational models have overlapping expressiveness, motivating a common programming model that affords unified reasoning and algorithm design. At the logical level we propose Lara, a lean algebra of three operators, that expresses RA and LA as well as relevant optimization rules. We show a series of proofs that position Lara %formal and informal at just the right level of expressiveness for a middleware algebra: more explicit than MapReduce but more general than RA or LA. At the physical level we find that the Lara operators afford efficient implementations using a single primitive that is available in a variety of backend engines: range scans over partitioned sorted maps. To evaluate these ideas, we implemented the Lara operators as range iterators in Apache Accumulo, a popular implementation of Google's BigTable. First we show how Lara expresses a sensor quality control task, and we measure the performance impact of optimizations Lara admits on this task. Second we show that the LaraDB implementation outperforms Accumulo's native MapReduce integration on a core task involving join and aggregation in the form of matrix multiply, especially at smaller scales that are typically a poor fit for scale-out approaches. We find that LaraDB offers a conceptually lean framework for optimizing mixed-abstraction analytics tasks, without giving up fast record-level updates and scans.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑