大数据集上重叠和多分辨率聚类算法的精度评估
Accuracy Evaluation of Overlapping and Multi-resolution Clustering Algorithms on Large Datasets
摘要:
聚类算法的性能进行评估的准确性指标的帮助下。聚类算法有很多种,它们是许多数据分析和探索系统的关键组成部分。然而,在大数据集上,重叠和多分辨率聚类算法的准确性度量很少。在本文中,我们首先讨论现有的度量,他们如何满足一组正式的约束,以及如何将它们应用到特定的情况下。然后,我们提出了一些优化和扩展这些指标。更具体地说,我们引入了一种新的索引技术,以减少平均F1得分评估的运行时间和内存复杂性。我们的技术可以应用于大型数据集,并且在单个CPU上比在高性能服务器上运行的最先进的实现更快。此外,我们提出了几个扩展所讨论的指标,以提高其有效性和满意度的正式约束,而不影响其效率。本文中讨论的所有指标都是用C++实现的,并且作为开源软件包免费提供,这些软件包既可以用作独立工具,也可以用作基准测试系统的一部分,以比较各种聚类算法。
英文摘要:
Performance of clustering algorithms is evaluated with the help of accuracy metrics. There is a great diversity of clustering algorithms, which are key components of many data analysis and exploration systems. However, there exist only few metrics for the accuracy measurement of overlapping and multi-resolution clustering algorithms on large datasets. In this paper, we first discuss existing metrics, how they satisfy a set of formal constraints, and how they can be applied to specific cases. Then, we propose several optimizations and extensions of these metrics. More specifically, we introduce a new indexing technique to reduce both the runtime and the memory complexity of the Mean F1 score evaluation. Our technique can be applied on large datasets and it is faster on a single CPU than state-of-the-art implementations running on high-performance servers. In addition, we propose several extensions of the discussed metrics to improve their effectiveness and satisfaction to formal constraints without affecting their efficiency. All the metrics discussed in this paper are implemented in C++ and are available for free as open-source packages that can be used either as stand-alone tools or as part of a benchmarking system to compare various clustering algorithms.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。