arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.03979cs.LG
收藏

二元分类器的零失败测试

Zero-failure testing of binary classifiers

Durham University(杜伦大学);Serve Legal Ltd(Serve Legal有限公司)

Ioannis Ivrissimtzis, Matthew Houliston, Shauna Concannon, Graham Roberts

AI总结:

本文提出基于零失败测试的二元分类器评估方法,通过非对称处理两类错误并构建难度递增的嵌套测试序列,以负样本成功率衡量性能,并在法定年龄估计场景中验证其有效性。

摘要:

我们提出使用源自零失败测试的性能指标来评估二元分类器。所提方法的主要特征是对两类错误进行非对称处理。具体而言,我们构建一个由正样本和负样本组成的测试集,将二元分类器的工作点设定为能够正确分类所有正样本的最低值,并以算法在负样本上的成功率作为性能度量。所提方法的一个特性使其区别于其他常用测试方法,即它允许构建一系列难度递增的测试,这些测试对应于正样本测试集的嵌套序列。我们在年龄估计问题上对所提方法进行了说明,该问题用于判断受试者是否超过法定年龄阈值,这一问题体现了两类错误的非对称性。实际上,将未达法定年龄的受试者错误分类属于法律和监管问题,而将超过法定年龄的人错误分类则主要是涉及年龄估计系统商业用户的效率问题。

英文摘要:

We propose using performance metrics derived from zero-failure testing to assess binary classifiers. The principal characteristic of the proposed approach is the asymmetric treatment of the two types of error. In particular, we construct a test set consisting of positive and negative samples, set the operating point of the binary classifier at the lowest value that will result to correct classifications of all positive samples, and use the algorithm's success rate on the negative samples as a performance measure. A property of the proposed approach, setting it apart from other commonly used testing methods, is that it allows the construction of a series of tests of increasing difficulty, corresponding to a nested sequence of positive sample test sets. We illustrate the proposed method on the problem of age estimation for determining whether a subject is above a legal age threshold, a problem that exemplifies the asymmetry of the two types of error. Indeed, misclassifying an under-aged subject is a legal and regulatory issue, while misclassifications of people above the legal age is an efficiency issue primarily concerning the commercial user of the age estimation system.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑