评估机器学习预测的敏感性：一种应用于金融知识的新方法
Assessing Sensitivity of Machine Learning Predictions.A Novel Toolbox with an Application to Financial Literacy
摘要：
尽管机器学习预测很受欢迎，但它对潜在的未观察到的预测因素很敏感。本文提出了一个通用的算法，评估如何省略一个未观察到的变量具有高解释力可能会影响模型的预测。此外，该算法将机器学习的使用从逐点预测扩展到推理和灵敏度分析。在应用中，我们展示了如何将该框架应用于具有固有不确定性的数据，例如学生在金融知识标准化评估中的分数。首先，使用贝叶斯加性回归树（BART），我们预测学生的财务素养分数（FLS）的学生与失踪FLS的一个小组。然后，我们通过比较具有和不具有高度解释性的合成预测因子的模型的预测和性能来评估预测的敏感性。我们没有发现增强的预测和性能有显著差异（即，具有合成预测器的模型）和原始模型。这一证据揭示了应用程序中使用的预测模型的稳定性。所提出的方法可以使用，以上和超越我们的激励实证的例子，在广泛的机器学习应用在社会和健康科学。
英文摘要：
Despite their popularity, machine learning predictions are sensitive to potential unobserved predictors. This paper proposes a general algorithm that assesses how the omission of an unobserved variable with high explanatory power could affect the predictions of the model. Moreover, the algorithm extends the usage of machine learning from pointwise predictions to inference and sensitivity analysis. In the application, we show how the framework can be applied to data with inherent uncertainty, such as students' scores in a standardized assessment on financial literacy. First, using Bayesian Additive Regression Trees (BART), we predict students' financial literacy scores (FLS) for a subgroup of students with missing FLS. Then, we assess the sensitivity of predictions by comparing the predictions and performance of models with and without a highly explanatory synthetic predictor. We find no significant difference in the predictions and performances of the augmented (i.e., the model with the synthetic predictor) and original model. This evidence sheds a light on the stability of the predictive model used in the application. The proposed methodology can be used, above and beyond our motivating empirical example, in a wide range of machine learning applications in social and health sciences.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
