arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1711.01034cs.DBcs.DC
收藏

PS-DBSCAN:基于AI(PAI)平台的高效并行DBSCAN算法

PS-DBSCAN: An Efficient Parallel DBSCAN Algorithm Based on Platform Of AI (PAI)

Xu Hu, Jun Huang, Minghui Qiu, Cen Chen, Wei Chu

摘要:

提出了一种基于不交集数据结构和参数服务器框架的并行DBSCAN算法PS-DBSCAN。由于同一个集群中的数据点可能分布在不同的工作节点上,从而产生多个不相交集,因此合并它们会产生很大的通信开销。在我们的算法中,我们采用了一种快速的全球联盟的方法来联盟的不相交集,以减轻通信负担。在不同规模数据集上的实验表明,PS-DBSCAN的通信效率比PDSDBSCAN提高了2-10倍。我们已经在阿里云的AI平台(PAI -https://pai.base.shuju.aliyun.com/)中发布了我们的PS-DBSCAN。我们还演示了如何在PAI中使用该方法。

英文摘要:

We present PS-DBSCAN, a communication efficient parallel DBSCAN algorithm that combines the disjoint-set data structure and Parameter Server framework in Platform of AI (PAI). Since data points within the same cluster may be distributed over different workers which result in several disjoint-sets, merging them incurs large communication costs. In our algorithm, we employ a fast global union approach to union the disjoint-sets to alleviate the communication burden. Experiments over the datasets of different scales demonstrate that PS-DBSCAN outperforms the PDSDBSCAN with 2-10 times speedup on communication efficiency. We have released our PS-DBSCAN in an algorithm platform called Platform of AI (PAI - https://pai.base.shuju.aliyun.com/) in Alibaba Cloud. We have also demonstrated how to use the method in PAI.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑