量化评价测度的公理化方法
Evaluation Measures for Quantification: An Axiomatic Approach
摘要:
量化是指在给定一组未标记项目$σ$和一组类别$\mathcal{C}=\{c_{1}, \ldots, c_{|\mathcal{C}|}\}$的情况下,估计每个类别$c_{i}\in \mathcal{C}$在$σ$中的流行率(或“相对频率”)。虽然原则上可以通过对$σ$中的每个项目进行分类并计算有多少此类项目已经用$c_{i}$标记来解决量化问题,但长期以来一直表明,这种“分类和计数”(CC)方法产生的量化准确度不佳。因此,量化不再被认为是分类的副产品,而是作为其自身的一项任务而发展。虽然科学界已经投入了大量的精力来设计更精确的量化方法,但它并没有投入太多的精力来讨论\emph{evaluation measure for quantification}(EMQ)应该享有哪些属性,以及应该采用哪些EMQ。本文列出了一些有趣的属性,一个EMQ可能会或可能不会享受,讨论如果(以及何时)这些属性是可取的,调查的EMQ已被使用到目前为止,并讨论他们是否享受或不上述属性。作为这项调查的结果,一些文献中使用的EMQ被证明是严重不合适的,而另一些则更接近量化社区的实际需要。然而,一个重要的结果是,没有现有的EMQ满足所有的属性确定为理想的,从而表明,需要更多的研究,以确定(或合成)一个真正足够的EMQ。
英文摘要:
Quantification is the task of estimating, given a set $σ$ of unlabelled items and a set of classes $\mathcal{C}=\{c_{1}, \ldots, c_{|\mathcal{C}|}\}$, the prevalence (or `relative frequency') in $σ$ of each class $c_{i}\in \mathcal{C}$. While quantification may in principle be solved by classifying each item in $σ$ and counting how many such items have been labelled with $c_{i}$, it has long been shown that this `classify and count' (CC) method yields suboptimal quantification accuracy. As a result, quantification is no longer considered a mere byproduct of classification, and has evolved as a task of its own. While the scientific community has devoted a lot of attention to devising more accurate quantification methods, it has not devoted much to discussing what properties an \emph{evaluation measure for quantification} (EMQ) should enjoy, and which EMQs should be adopted as a result. This paper lies down a number of interesting properties that an EMQ may or may not enjoy, discusses if (and when) each of these properties is desirable, surveys the EMQs that have been used so far, and discusses whether they enjoy or not the above properties. As a result of this investigation, some of the EMQs that have been used in the literature turn out to be severely unfit, while others emerge as closer to what the quantification community actually needs. However, a significant result is that no existing EMQ satisfies all the properties identified as desirable, thus indicating that more research is needed in order to identify (or synthesize) a truly adequate EMQ.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。