arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1401.4590cs.AIcs.LG
收藏

基于一致改进比的组合评价算法及其在聚类中的应用

Combining Evaluation Metrics via the Unanimous Improvement Ratio and its Application to Clustering Tasks

UNED(西班牙国立远程教育大学);Queens College (CUNY)(皇后学院(纽约市立大学))

Enrique Amigó, Julio Gonzalo, Javier Artiles, Felisa Verdejo

摘要:

许多人工智能任务不能用单一的质量标准进行评估,需要某种加权组合来提供系统排名。加权组合度量的一个问题是,相对权重的微小变化可能会导致系统排名的重大变化。本文介绍了一致改进比(UIR),一个措施,补充标准的度量组合标准(如范Rijsbergen的F-措施),并表明如何强大的测量差异是在各个指标的相对权重的变化。UIR旨在阐明两个系统之间的感知差异是否是如何加权各个指标的人为因素。除了讨论UIR的理论基础,本文提出了实证结果,确认的有效性和实用性的度量文本聚类问题,其中有一个权衡精度和召回率为基础的指标和结果是特别敏感的加权方案,用于结合它们。值得注意的是,我们的实验表明,UIR可以被用来作为一个给定的测试床上测量的系统之间的差异如何也将在不同的测试床的预测。

英文摘要:

Many Artificial Intelligence tasks cannot be evaluated with a single quality criterion and some sort of weighted combination is needed to provide system rankings. A problem of weighted combination measures is that slight changes in the relative weights may produce substantial changes in the system rankings. This paper introduces the Unanimous Improvement Ratio (UIR), a measure that complements standard metric combination criteria (such as van Rijsbergen's F-measure) and indicates how robust the measured differences are to changes in the relative weights of the individual metrics. UIR is meant to elucidate whether a perceived difference between two systems is an artifact of how individual metrics are weighted. Besides discussing the theoretical foundations of UIR, this paper presents empirical results that confirm the validity and usefulness of the metric for the Text Clustering problem, where there is a tradeoff between precision and recall based metrics and results are particularly sensitive to the weighting scheme used to combine them. Remarkably, our experiments show that UIR can be used as a predictor of how well differences between systems measured on a given test bed will also hold in a different test bed.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑