Minimizing expected loss measured by a proper scoring rule, such as Brier score or log-loss (cross-entropy), is a common objective while training a probabilistic classifier. If the data have experienced dataset shift where the class distributions change post-training, then often the model's performance will decrease, over-estimating the probabilities of some classes while under-estimating the others on average. We propose unbounded and bounded general adjustment (UGA and BGA) methods that transform all predictions to (re-)equalize the average prediction and the class distribution. These methods act differently depending on which proper scoring rule is to be minimized, and we have a theoretical guarantee of reducing loss on test data, if the exact class distribution is known. We also demonstrate experimentally that, when in practice the class distribution is known only approximately, there is often still a reduction in loss depending on the amount of shift and the precision to which the class distribution is known.
Deep Neural Network Calibration by Reducing Classifier Shift with Stochastic Masking
School of Artificial Intelligence, Jilin University(吉林大学人工智能学院);CSIRO's Data61 and Monash University (adjunct)(CSIRO的数据61与墨尔本大学(兼职));King Abdullah University of Science and Technology(国王阿卜杜勒阿齐兹科技大学)
German Research Center for Artificial intelligence (DFKI)(德国人工智能研究中心);University of Bologna(博洛尼亚大学);Istituto Nazionale di Fisica Nucleare (INFN)(意大利国家核物理研究院)
FinGAN: Generative Adversarial Network for Analytical Customer Relationship Management in Banking and Insurance
Institute for Development and Research in Banking Technology(银行技术发展与研究所);University of Hyderabad(海得拉巴大学);Mathworks(MathWorks公司)
信用卡客户流失预测、保险欺诈检测和贷款违约预测是分析型客户关系管理中的重要问题。由于欺诈、流失和违约发生的频率较低,这些问题的数据集自然是高度不平衡的。因此,所有监督机器学习分类器在这种不平衡的数据集上训练时往往会产生大量的误报率。我们提出了两种数据平衡的方法。首先,我们提出了一种过采样的方法来产生合成样本的少数类生成对抗网络(GAN)。我们分别采用Vanilla GAN [1]、Wasserstein GAN [2]和CT GAN [3]对少数类样本进行过采样。为了评估我们所提出的方法的有效性,我们使用了许多机器学习分类器,包括随机森林,决策树,支持向量机(SVM)和逻辑回归对GANs平衡的数据。在第二种方法中,我们引入了一种混合方法来处理数据不平衡。在第二种方式中,我们通过使用一类支持活力机(OCSVM)获得的欠采样多数类数据来增强GAN过采样的合成少数类数据,从而同时利用欠采样和过采样的能力[4]。我们结合了GAN生成的过采样数据和OCSVM [4]欠采样的数据,并将结果数据传递给分类器。当我们将我们的结果与Farquad等人的结果进行比较时。[5],Sundarkumar,Ravi和Siddeshwar [6],我们提出的方法在所有数据集的ROC曲线下面积(AUC)方面优于以前的结果。