基于MapReduce框架的Apriori算法综述
Review of Apriori Based Algorithms on MapReduce Framework
摘要:
挖掘频繁项集的Apriori算法是最流行和最广泛使用的数据采掘算法之一。为了提高Apriori算法的性能,目前已经提出了许多并行和分布式平台上的算法。它们在负载均衡技术、存储系统、数据分解技术和数据布局等方面存在差异。大多数分布式框架的问题是管理分布式系统的开销和缺乏高级并行程序设计语言。而且,对于网格计算,总是存在节点故障的潜在机会,这会导致任务的多次重新执行。这些问题可以通过Google引入的MapReduce框架来解决。MapReduce是一种高效,可扩展和简化的编程模型,用于大型商用计算机集群上的大规模分布式数据处理,也用于云计算。本文介绍了在MapReduce框架上实现的并行Apriori算法。它们被分类的基础上的映射和减少功能,用于实现它们,例如1阶段与k阶段,I/O的映射器,组合器和还原器,使用功能的组合器内映射器等,本次调查讨论和分析的各种实现Apriori的MapReduce框架的基础上,他们的显着特点。此外,还包括MapReduce框架的优点和局限性。
英文摘要:
The Apriori algorithm that mines frequent itemsets is one of the most popular and widely used data mining algorithms. Now days many algorithms have been proposed on parallel and distributed platforms to enhance the performance of Apriori algorithm. They differ from each other on the basis of load balancing technique, memory system, data decomposition technique and data layout used to implement them. The problems with most of the distributed framework are overheads of managing distributed system and lack of high level parallel programming language. Also with grid computing there is always potential chances of node failures which cause multiple re-executions of tasks. These problems can be overcome by the MapReduce framework introduced by Google. MapReduce is an efficient, scalable and simplified programming model for large scale distributed data processing on a large cluster of commodity computers and also used in cloud computing. In this paper, we present the overview of parallel Apriori algorithm implemented on MapReduce framework. They are categorized on the basis of Map and Reduce functions used to implement them e.g. 1-phase vs. k-phase, I/O of Mapper, Combiner and Reducer, using functionality of Combiner inside Mapper etc. This survey discusses and analyzes the various implementations of Apriori on MapReduce framework on the basis of their distinguishing characteristics. Moreover, it also includes the advantages and limitations of MapReduce framework.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。