arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1209.4257cs.DBcs.DC
收藏

通信高效和精确的分布式流数据聚类

Communication-Efficient and Exact Clustering Distributed Streaming Data

Dang-Hoan Tran

摘要:

一个广泛使用的方法来聚类一个单一的数据流是两个阶段的方法,其中在线阶段创建和维护微集群,而离线阶段从微集群生成宏集群。我们使用这种方法,提出了一个分布式框架聚类流数据。我们提出的框架由基本过程组成:一个协调器-站点过程和许多远程站点过程。远程站点进程可以直接与协调器进程通信,但不能与其他远程站点进程通信。每个远程站点进程从其本地数据流中生成并维护表示簇信息摘要的微簇。远程站点通过串行化技术将本地微聚类发送给协调器,或者协调器调用远程方法从远程站点获取本地微聚类。协调器接收到来自远程站点的所有局部微聚类后,通过宏聚类方法生成全局聚类。我们的理论和实证结果表明,我们的分布式框架产生的全局聚类是类似的底层集中式算法在同一数据集上产生的聚类。通过使用局部微聚类方法,我们的框架实现了高可扩展性和通信效率。

英文摘要:

A widely used approach to clustering a single data stream is the two-phased approach in which the online phase creates and maintains micro-clusters while the off-line phase generates the macro-clustering from the micro-clusters. We use this approach to propose a distributed framework for clustering streaming data. Our proposed framework consists of fundamen- tal processes: one coordinator-site process and many remote-site processes. Remote-site processes can directly communicate with the coordinator-process but cannot communicate the other remote site processes. Every remote-site process generates and maintains micro- clusters that represent cluster information summary, from its local data stream. Remote sites send the local micro-clusterings to the coordinator by the serialization technique, or the coordinator invokes the remote methods in order to get the local micro-clusterings from the remote sites. After the coordinator receives all the local micro-clusterings from the remote sites, it generates the global clustering by the macro-clustering method. Our theoretical and empirical results show that, the global clustering generated by our distributed framework is similar to the clustering generated by the underlying centralized algorithm on the same data set. By using the local micro-clustering approach, our framework achieves high scalability, and communication-efficiency.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑