arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1403.7394cs.DC
收藏

基于MapReduce的并行层次仿射传播算法

Parallel Hierarchical Affinity Propagation with MapReduce

Dillon Mark Rose, Jean Michel Rouly, Rana Haber, Nenad Mijatovic, Adrian M. Peter

摘要:

互联网和社交媒体的加速发展和爆炸正在产生大量的数据(在zettabyte规模上)。从海量大数据中操纵和提取可操作情报的愿望中,最重要的是需要可扩展的、性能敏感的分析算法。为了直接满足这一需求,我们提出了一种新的MapReduce实现的基于样本的聚类算法称为亲和传播。我们的并行化策略扩展到多层次的层次亲和传播算法,使分层聚合的非结构化数据与最小的自由参数,原则上只需要一个数据点之间的相似性度量。我们详细介绍了我们的方法的线性运行时的复杂性,克服了原来的算法的限制二次复杂性。在各种合成和真实数据集(例如图像和点数据)上对我们的聚类方法进行了实验验证,证明了我们对其他最先进的MapReduce聚类技术的竞争力。

英文摘要:

The accelerated evolution and explosion of the Internet and social media is generating voluminous quantities of data (on zettabyte scales). Paramount amongst the desires to manipulate and extract actionable intelligence from vast big data volumes is the need for scalable, performance-conscious analytics algorithms. To directly address this need, we propose a novel MapReduce implementation of the exemplar-based clustering algorithm known as Affinity Propagation. Our parallelization strategy extends to the multilevel Hierarchical Affinity Propagation algorithm and enables tiered aggregation of unstructured data with minimal free parameters, in principle requiring only a similarity measure between data points. We detail the linear run-time complexity of our approach, overcoming the limiting quadratic complexity of the original algorithm. Experimental validation of our clustering methodology on a variety of synthetic and real data sets (e.g. images and point data) demonstrates our competitiveness against other state-of-the-art MapReduce clustering techniques.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑