arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2108.05677cs.LGcs.AI
收藏

数据集的非一致性函数和难度如何影响共形分类器的效率

How Nonconformity Functions and Difficulty of Datasets Impact the Efficiency of Conformal Classifiers

University of Luxembourg(卢森堡大学)

Marharyta Aleksandrova, Oleg Chertov

摘要:

保形预测的属性,以保证所需的准确率,使这个框架在各种实际应用中的吸引力。然而,这种特性是以精度降低为代价的。在保形分类的情况下,系统可以输出多个类别标签而不是一个。从文献中还可以看出,非一致性函数的选择对共形分类器的效率有重要影响。最近,它表明,不同的模型不可知的非一致性功能的结果在保形分类器具有不同的特点。对于基于神经网络的共形分类器,逆概率(或铰链损失)允许最小化预测标签的平均数量,并且余量导致更大比例的单例预测。在这项工作中,我们的目标是进一步扩展这项研究。我们使用8种不同的分类算法进行了实验评估,并讨论了先前观察到的关系是否成立。此外,我们提出了一个成功的方法来结合这两个非一致性函数的属性。使用11个真实数据集和5个合成数据集进行实验评估。

英文摘要:

The property of conformal predictors to guarantee the required accuracy rate makes this framework attractive in various practical applications. However, this property is achieved at a price of reduction in precision. In the case of conformal classification, the systems can output multiple class labels instead of one. It is also known from the literature, that the choice of nonconformity function has a major impact on the efficiency of conformal classifiers. Recently, it was shown that different model-agnostic nonconformity functions result in conformal classifiers with different characteristics. For a Neural Network-based conformal classifier, the inverse probability (or hinge loss) allows minimizing the average number of predicted labels, and margin results in a larger fraction of singleton predictions. In this work, we aim to further extend this study. We perform an experimental evaluation using 8 different classification algorithms and discuss when the previously observed relationship holds or not. Additionally, we propose a successful method to combine the properties of these two nonconformity functions. The experimental evaluation is done using 11 real and 5 synthetic datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑