量化方法的比较评估
A Comparative Evaluation of Quantification Methods
AI总结:
本文通过对24种量化方法在40多个数据集上的二分类和多分类场景实证比较,填补了量化算法选择的研究空白,确定了不同场景下的优秀方法,为从业者和研究者提供指导。
摘要:
量化是指估计未见过的数据上类别标签分布的问题,也是监督机器学习中一个不断发展的研究领域,近年来已提出多种不同的算法。然而,目前尚无支持算法选择的量化方法综合实证比较。本研究通过对24种不同量化方法在40多个数据集上进行全面的实证性能比较,填补了这一研究空白,考虑了二分类和多分类量化场景。我们发现没有单一算法能普遍优于所有竞争者,但在二分类场景中确定了一组表现最佳的方法,包括基于阈值选择的Median Sweep和TSMax方法、包含HDy方法的DyS框架、Forman的混合模型以及Friedman方法。在多分类场景中,我们观察到另一组广泛的算法表现良好,包括HDx方法、广义概率调整计数(Generalized Probabilistic Adjusted Count)、readme方法、能量距离最小化方法、量化的EM算法以及Friedman方法。我们还发现,在大多数情况下,调整底层分类器对量化性能的影响有限。更普遍地说,我们发现多分类量化的性能不如二分类场景中获得的结果。我们的结果可为打算应用量化算法的从业者提供指导,并帮助研究人员确定未来研究的机会。
英文摘要:
Quantification represents the problem of estimating the distribution of class labels on unseen data. It also represents a growing research field in supervised machine learning, for which a large variety of different algorithms has been proposed in recent years. However, a comprehensive empirical comparison of quantification methods that supports algorithm selection is not available yet. In this work, we close this research gap by conducting a thorough empirical performance comparison of 24 different quantification methods on overall more than 40 data sets, considering binary as well as multiclass quantification settings. We observe that no single algorithm generally outperforms all competitors, but identify a group of methods including the threshold selection-based Median Sweep and TSMax methods, the DyS framework including the HDy method, Forman's mixture model, and Friedman's method that performs best in the binary setting. For the multiclass setting, we observe that a different, broad group of algorithms yields good performance, including the HDx method, the Generalized Probabilistic Adjusted Count, the readme method, the energy distance minimization method, the EM algorithm for quantification, and Friedman's method. We also find that tuning the underlying classifiers has in most cases only a limited impact on the quantification performance. More generally, we find that the performance on multiclass quantification is inferior to the results obtained in the binary setting. Our results can guide practitioners who intend to apply quantification algorithms and help researchers to identify opportunities for future research.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。