arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1208.4165cs.DB
收藏

MADlib分析库或MAD技能,SQL

The MADlib Analytics Library or MAD Skills, the SQL

Joe Hellerstein, Christopher Ré, Florian Schoppmann, Daisy Zhe Wang, Eugene Fratkin, Aleksander Gorajek, Kee Siong Ng, Caleb Welton, Xixuan Feng, Kun Li, Arun Kumar

摘要:

MADlib是一个免费的开源数据库内分析方法库。它为机器学习、数据采掘和统计提供了一套不断发展的基于SQL的算法,这些算法在数据库引擎中大规模运行,无需将数据导入/导出到其他工具。MADlib的目标是最终为可扩展的数据库系统服务,类似于R的CRAN库:一个统计方法的社区存储库,这次编写时考虑了规模和并行性。在本文中,我们介绍了MADlib项目,包括导致其开始的背景,以及其开源性质的动机。我们提供了库的体系结构和设计模式的概述,并提供了在这种情况下的各种统计方法的描述。我们在一个中等规模的测试群集上,通过Greenplum并行DBMS,对其中一种方法的核心设计模式的性能和加速结果进行了测试。然后,我们报告了将学术研究纳入MADlib的两个初步努力,这是该项目的目标之一。MADlib可在http://madlib.net上免费获得,该项目对新方法和其他数据库平台的端口开放。

英文摘要:

MADlib is a free, open source library of in-database analytic methods. It provides an evolving suite of SQL-based algorithms for machine learning, data mining and statistics that run at scale within a database engine, with no need for data import/export to other tools. The goal is for MADlib to eventually serve a role for scalable database systems that is similar to the CRAN library for R: a community repository of statistical methods, this time written with scale and parallelism in mind. In this paper we introduce the MADlib project, including the background that led to its beginnings, and the motivation for its open source nature. We provide an overview of the library's architecture and design patterns, and provide a description of various statistical methods in that context. We include performance and speedup results of a core design pattern from one of those methods over the Greenplum parallel DBMS on a modest-sized test cluster. We then report on two initial efforts at incorporating academic research into MADlib, which is one of the project's goals. MADlib is freely available at http://madlib.net, and the project is open for contributions of both new methods, and ports to additional database platforms.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑