Blaze：简化的高性能集群计算
Blaze: Simplified High Performance Cluster Computing
摘要：
MapReduce及其变体大大简化和加速了开发并行程序的过程。然而，大多数MapReduce实现都集中在数据密集型任务上，而许多现实世界的任务都是计算密集型的，它们的数据可以分布到内存中。对于这些任务，MapReduce程序的速度可能比手动优化的程序慢得多。我们目前的火焰，一个C++库，可以很容易地开发高性能的并行程序，这样的计算密集型任务。Blaze的核心是一个高度优化的内存中MapReduce函数，与传统的MapReduce实现相比，它有三个主要的改进：快速减少，快速序列化和对小的固定键范围的特殊处理。我们还提供额外的便利，使开发并行程序类似于开发串行程序。这些改进使Blaze成为一个易于使用的集群计算库，接近手动优化并行代码的速度。我们将Blaze应用于一些常见的数据采掘任务，包括词频计数，PageRank，k-means，期望最大化（高斯混合模型）和k-最近邻。在这些任务中，Blaze的平均性能比Apache Spark高出10倍以上，而且Blaze的速度几乎与节点数量成线性关系。此外，Blaze在其实现中仅使用MapReduce函数和3个实用程序函数，而Spark在其官方实现中使用了近30种不同的并行原语。
英文摘要：
MapReduce and its variants have significantly simplified and accelerated the process of developing parallel programs. However, most MapReduce implementations focus on data-intensive tasks while many real-world tasks are compute intensive and their data can fit distributedly into the memory. For these tasks, the speed of MapReduce programs can be much slower than those hand-optimized ones. We present Blaze, a C++ library that makes it easy to develop high performance parallel programs for such compute intensive tasks. At the core of Blaze is a highly-optimized in-memory MapReduce function, which has three main improvements over conventional MapReduce implementations: eager reduction, fast serialization, and special treatment for a small fixed key range. We also offer additional conveniences that make developing parallel programs similar to developing serial programs. These improvements make Blaze an easy-to-use cluster computing library that approaches the speed of hand-optimized parallel code. We apply Blaze to some common data mining tasks, including word frequency count, PageRank, k-means, expectation maximization (Gaussian mixture model), and k-nearest neighbors. Blaze outperforms Apache Spark by more than 10 times on average for these tasks, and the speed of Blaze scales almost linearly with the number of nodes. In addition, Blaze uses only the MapReduce function and 3 utility functions in its implementation while Spark uses almost 30 different parallel primitives in its official implementation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
