arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2103.03788cs.LGstat.ML
收藏

损失估计器改进模型泛化

Loss Estimators Improve Model Generalization

Arizona State University(亚利桑那州立大学);Lawrence Livermore National Laboratory(劳伦斯利弗莫尔国家实验室);IBM Research Almaden(IBM阿尔马登研究中心)

Vivek Narayanaswamy, Jayaraman J. Thiagarajan, Deepta Rajan, Andreas Spanias

摘要:

随着人们对采用人工智能方法进行临床诊断的兴趣越来越大,安全部署此类工具的关键一步是确保模型不仅能产生准确的预测,而且不会推广到训练数据无法提供有意义证据的数据体系。现有的方法,以确保模型预测的分布是类似的真实分布依赖于显式的不确定性估计,本质上是难以校准。在本文中,我们提出了训练损失估计器旁边的预测模型,使用对比训练目标,直接估计预测的不确定性。有趣的是,我们发现,除了产生校准良好的不确定性,这种方法提高了预测的泛化行为。使用皮肤病学用例,我们展示了损失估计量对模型泛化的影响,包括其对分布数据的保真度以及检测分布样本或训练过程中看不到的新类的能力。

英文摘要:

With increased interest in adopting AI methods for clinical diagnosis, a vital step towards safe deployment of such tools is to ensure that the models not only produce accurate predictions but also do not generalize to data regimes where the training data provide no meaningful evidence. Existing approaches for ensuring the distribution of model predictions to be similar to that of the true distribution rely on explicit uncertainty estimators that are inherently hard to calibrate. In this paper, we propose to train a loss estimator alongside the predictive model, using a contrastive training objective, to directly estimate the prediction uncertainties. Interestingly, we find that, in addition to producing well-calibrated uncertainties, this approach improves the generalization behavior of the predictor. Using a dermatology use-case, we show the impact of loss estimators on model generalization, in terms of both its fidelity on in-distribution data and its ability to detect out of distribution samples or new classes unseen during training.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑