arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2111.02529cs.LG
收藏

发生转变:调整分类器

Shift Happens: Adjusting Classifiers

University of Tartu(塔尔图大学)

Theodore James Thibault Heiser, Mari-Liis Allikivi, Meelis Kull

摘要:

在训练概率分类器时,最小化通过适当的评分规则(例如Brier评分或对数损失(交叉熵))测量的预期损失是常见的目标。如果数据在训练后经历了类分布变化的数据集偏移,那么模型的性能通常会下降,高估了某些类的概率,而低估了其他类的概率。我们提出了无界和有界的一般调整(UGA和BGA)方法,将所有的预测(重新)均衡的平均预测和类分布。这些方法的作用不同,这取决于适当的评分规则是最小化,我们有一个理论上的保证,减少损失的测试数据,如果确切的类分布是已知的。我们还通过实验证明,当在实践中的类分布是已知的只有大约,往往仍然是减少损失取决于量的移位和精度的类分布是已知的。

英文摘要:

Minimizing expected loss measured by a proper scoring rule, such as Brier score or log-loss (cross-entropy), is a common objective while training a probabilistic classifier. If the data have experienced dataset shift where the class distributions change post-training, then often the model's performance will decrease, over-estimating the probabilities of some classes while under-estimating the others on average. We propose unbounded and bounded general adjustment (UGA and BGA) methods that transform all predictions to (re-)equalize the average prediction and the class distribution. These methods act differently depending on which proper scoring rule is to be minimized, and we have a theoretical guarantee of reducing loss on test data, if the exact class distribution is known. We also demonstrate experimentally that, when in practice the class distribution is known only approximately, there is often still a reduction in loss depending on the amount of shift and the precision to which the class distribution is known.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑