arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1702.05425cs.DScs.DB
收藏

线性时间上的精确聚类

Exact clustering in linear time

Jonathan A. Marshall, Lawrence C. Rafsky

摘要:

数据聚类的时间复杂度基本上是二次的,随着数据项的数量而变慢,因为每个项都与前面的项进行相似性比较。如果不采用概率方法或限制聚类的数量,大型数据集的聚类是不可行的。在这里,我们介绍MIMOSA,一类新的算法,实现线性时间的计算复杂性聚类任务。MIMOSA算法标记并匹配哈希表中的部分签名密钥,以获得准确、无错误的簇检索。通过对10,000,000篇新闻文章的数据集按新闻主题进行聚类的基准测量发现,MIMOSA实现比标准质心实现快四个数量级以上。

英文摘要:

The time complexity of data clustering has been viewed as fundamentally quadratic, slowing with the number of data items, as each item is compared for similarity to preceding items. Clustering of large data sets has been infeasible without resorting to probabilistic methods or to capping the number of clusters. Here we introduce MIMOSA, a novel class of algorithms which achieve linear time computational complexity on clustering tasks. MIMOSA algorithms mark and match partial-signature keys in a hash table to obtain exact, error-free cluster retrieval. Benchmark measurements, on clustering a data set of 10,000,000 news articles by news topic, found that a MIMOSA implementation finished more than four orders of magnitude faster than a standard centroid implementation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑