arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1406.4923cs.DBastro-ph.IMcs.CEcs.DCcs.MS
收藏

使用Accumulo和D4M实现每秒1亿次数据库插入

Achieving 100,000,000 database inserts per second using Accumulo and D4M

Jeremy Kepner, William Arcand, David Bestor, Bill Bergeron, Chansup Byun, Vijay Gadepally, Matthew Hubbell, Peter Michaleas, Julie Mullen, Andrew Prout, Albert … 展开作者

Jeremy Kepner, William Arcand, David Bestor, Bill Bergeron, Chansup Byun, Vijay Gadepally, Matthew Hubbell, Peter Michaleas, Julie Mullen, Andrew Prout, Albert Reuther, Antonio Rosa, Charles Yee

摘要:

Apache Accumulo数据库是一个开源的宽松一致性数据库,广泛用于政府应用程序。Accumulo旨在为非结构化数据提供高性能,例如网络数据图。本文使用Graph500基准测试的数据测试了Accumulo的性能。动态分布式维度数据模型(D4M)软件用于在运行MIT SuperCloud软件堆栈的216节点集群上实现基准测试。达到了每秒超过100,000,000次数据库插入的峰值性能,这比之前发布的任何其他数据库的最高值大100倍。性能随接收客户端的数量、数据库服务器的数量和数据大小线性扩展。性能是通过采用几种超级计算技术来实现的:分布式阵列、区域分解、自适应负载均衡和单程序多数据编程。

英文摘要:

The Apache Accumulo database is an open source relaxed consistency database that is widely used for government applications. Accumulo is designed to deliver high performance on unstructured data such as graphs of network data. This paper tests the performance of Accumulo using data from the Graph500 benchmark. The Dynamic Distributed Dimensional Data Model (D4M) software is used to implement the benchmark on a 216-node cluster running the MIT SuperCloud software stack. A peak performance of over 100,000,000 database inserts per second was achieved which is 100x larger than the highest previously published value for any other database. The performance scales linearly with the number of ingest clients, number of database servers, and data size. The performance was achieved by adapting several supercomputing techniques to this application: distributed arrays, domain decomposition, adaptive load balancing, and single-program-multiple-data programming.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑