arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2202.04166stat.MEstat.ML
收藏

统计模型的故障检测、识别和重新拟合

The Lifecycle of a Statistical Model: Model Failure Detection, Identification, and Refitting

Alnur Ali, Maxime Cauchois, John C. Duchi

摘要:

多年来,统计机器学习社区在开发用于估计、预测和推理的高表达性工具方面表现出了相当大的智慧。这些发展背后的基本假设是,数据来自一个固定的人口,并显示很少的异质性。但现实要复杂得多:统计模型现在在发布到现实世界的系统和科学应用中时经常会失败,而这些假设很少成立。因此,我们追求不同的路径,在本文中相对于开发新的估计和预测方法的陈旧的线索。在本文中,我们开发了用于检测和识别模型性能开始下降的协变量空间(子群体)区域的工具和理论,并研究了通过改装进行干预以修复这些故障。我们用三个真实世界的数据集(包括一个涉及预测COVID-19发病率的时间序列)展示了实证结果,表明我们的方法产生了可解释的结果,对跟踪模型性能很有用,并且可以通过重新调整来提高模型性能。我们补充这些经验的结果与理论证明,我们的方法是最小极大最优的恢复异常亚群,以及改装,以提高准确性,在一个结构化的正常手段设置。

英文摘要:

The statistical machine learning community has demonstrated considerable resourcefulness over the years in developing highly expressive tools for estimation, prediction, and inference. The bedrock assumptions underlying these developments are that the data comes from a fixed population and displays little heterogeneity. But reality is significantly more complex: statistical models now routinely fail when released into real-world systems and scientific applications, where such assumptions rarely hold. Consequently, we pursue a different path in this paper vis-a-vis the well-worn trail of developing new methodology for estimation and prediction. In this paper, we develop tools and theory for detecting and identifying regions of the covariate space (subpopulations) where model performance has begun to degrade, and study intervening to fix these failures through refitting. We present empirical results with three real-world data sets -- including a time series involving forecasting the incidence of COVID-19 -- showing that our methodology generates interpretable results, is useful for tracking model performance, and can boost model performance through refitting. We complement these empirical results with theory proving that our methodology is minimax optimal for recovering anomalous subpopulations as well as refitting to improve accuracy in a structured normal means setting.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑