arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2209.01857stat.MLcs.LGstat.ME
收藏

基于广义随机占优的分类器统计比较

Statistical Comparisons of Classifiers by Generalized Stochastic Dominance

Ludwig-Maximilians-Universität(慕尼黑大学)

Christoph Jansen, Malte Nalenz, Georg Schollmeyer, Thomas Augustin

AI总结:

本文提出基于偏好系统和广义随机占优的分类器比较框架,通过线性规划与观测随机化检验,实现多数据集、多准则下的统计比较,避免依赖聚合指标。

摘要:

尽管对于机器学习算法的发展而言这是一个至关重要的问题,但关于如何在多个数据集上、针对多个准则对分类器进行比较,目前仍无共识。每一个比较框架都面临(至少)三个基本挑战:质量准则的多样性、数据集的多样性以及数据集选择的随机性。在本文中,我们通过采用决策理论的最新进展,为这场激烈的讨论增添了一个新的视角。基于所谓的偏好系统,我们的框架通过广义随机占优概念对分类器进行排序,有力地规避了对聚合指标的繁琐且常常自相矛盾的依赖。此外,我们表明广义随机占优可以通过求解易于处理的线性规划来操作化,并且还可以采用一种自适应的双样本观测随机化检验进行统计检验。这确实产生了一个强大的框架,能够在多个数据集上同时针对多个质量准则对分类器进行统计比较。我们在模拟研究中以及在一组标准基准数据集上展示并考察了我们的框架。

英文摘要:

Although being a crucial question for the development of machine learning algorithms, there is still no consensus on how to compare classifiers over multiple data sets with respect to several criteria. Every comparison framework is confronted with (at least) three fundamental challenges: the multiplicity of quality criteria, the multiplicity of data sets and the randomness of the selection of data sets. In this paper, we add a fresh view to the vivid debate by adopting recent developments in decision theory. Based on so-called preference systems, our framework ranks classifiers by a generalized concept of stochastic dominance, which powerfully circumvents the cumbersome, and often even self-contradictory, reliance on aggregates. Moreover, we show that generalized stochastic dominance can be operationalized by solving easy-to-handle linear programs and moreover statistically tested employing an adapted two-sample observation-randomization test. This yields indeed a powerful framework for the statistical comparison of classifiers over multiple data sets with respect to multiple quality criteria simultaneously. We illustrate and investigate our framework in a simulation study and with a set of standard benchmark data sets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑