所有的模型都是错的,但许多是有用的:通过同时研究整个类别的预测模型来学习变量的重要性
All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously
摘要:
变量重要性(VI)工具描述协变量对预测模型准确性的贡献程度。然而,对于一个性能良好的模型(例如,具有固定系数向量$β$的线性模型$f(\mathbf{x})=\mathbf{x}^{T}β$)来说重要的变量对于另一个模型来说可能不重要。在本文中,我们提出了模型类依赖(MCR)的VI值的范围在所有表现良好的模型在一个预先指定的类。因此,MCR给出了一个更全面的描述的重要性,占的事实,许多预测模型,可能不同的参数形式,可能适合的数据。在推导MCR的过程中,我们基于随机森林中使用的VI措施,展示了基于置换的VI估计的几个信息性结果。具体来说,我们推导出一个单一的预测模型,U-统计,条件变量的重要性,条件因果效应,和线性模型系数的排列重要性估计之间的连接。然后,我们给出概率界MCR,使用一种新的,可推广的技术。我们将MCR应用于布劳沃德县犯罪记录的公共数据集,以研究累犯预测模型对性别和种族的依赖。在此应用中,MCR可用于帮助通知VI未知的专有模型。
英文摘要:
Variable importance (VI) tools describe how much covariates contribute to a prediction model's accuracy. However, important variables for one well-performing model (for example, a linear model $f(\mathbf{x})=\mathbf{x}^{T}β$ with a fixed coefficient vector $β$) may be unimportant for another model. In this paper, we propose model class reliance (MCR) as the range of VI values across all well-performing model in a prespecified class. Thus, MCR gives a more comprehensive description of importance by accounting for the fact that many prediction models, possibly of different parametric forms, may fit the data well. In the process of deriving MCR, we show several informative results for permutation-based VI estimates, based on the VI measures used in Random Forests. Specifically, we derive connections between permutation importance estimates for a single prediction model, U-statistics, conditional variable importance, conditional causal effects, and linear model coefficients. We then give probabilistic bounds for MCR, using a novel, generalizable technique. We apply MCR to a public data set of Broward County criminal records to study the reliance of recidivism prediction models on sex and race. In this application, MCR can be used to help inform VI for unknown, proprietary models.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。