LOCO-AdaMP：面向自适应小批量集成增强预测的内置LOCO推断

中文摘要 AI 辅助

随着黑盒机器学习模型日益普遍，提取具有不确定性量化的解释已成为一项关键挑战。一种流行的解释类型是留一协变量（LOCO）特征重要性，而先前的LOCO推断方法通常需要数据分割或模型重拟合。近期提出的集成框架LOCO-MP通过使用同时对观测和特征进行子采样的小批量来解决这些挑战，但在高维稀疏场景中，大规模特征子采样可能损害预测性能。受此局限性的启发，我们考虑采用由LOCO重要性引导的自适应特征采样的小批量集成，并提出LOCO-AdaMP，它为所得的自适应小批量集成实现了免费的LOCO推断。我们证明，尽管自适应采样分布与LOCO重要性统计量之间存在复杂依赖关系，LOCO-AdaMP仍能产生显著改进的预测模型，同时无需数据分割即可保持渐近有效的特征重要性推断。我们的分析依赖于对迭代更新的采样概率的仔细的留二扰动界，以及由观测子采样引起的LOCO得分的稳定性。在合成和真实数据集上的实证结果展示了LOCO-AdaMP在预测性能、推断能力和稳定性方面优于现有方法的优势。总体而言，LOCO-AdaMP提供了一个灵活的集成框架（对基模型不可知），为回归任务同时提供强大的预测性能和渐近有效、有力的特征重要性推断。

英文摘要

As black-box machine learning models become increasingly common, extracting interpretations with uncertainty quantification has become a critical challenge. One popular type of interpretation is leave-one-covariate-out (LOCO) feature importance, while prior LOCO inference methods often require data-splitting or model-refitting. A recent ensemble framework, LOCO-MP, addresses these challenges using minipatches that subsample both observations and features, but massive feature subsampling can hurt prediction in high-dimensional sparse settings. Motivated by this limitation, we consider minipatch ensembles with adaptive feature sampling guided by LOCO importance, and propose LOCO-AdaMP, which enables free LOCO inference for the resulting adaptive minipatch ensemble. We show that LOCO-AdaMP yields substantially improved predictive models while retaining asymptotically valid feature importance inference without data-splitting, despite the complex dependence between the adaptive sampling distribution and the LOCO importance statistics. Our analysis relies on a careful leave-two-out perturbation bound for the iteratively updated sampling probabilities together with the stability of LOCO scores induced by observation subsampling. Empirical results on synthetic and real datasets demonstrate advantages of LOCO-AdaMP over existing methods in predictive performance, inferential power, and stability. Overall, LOCO-AdaMP provides a flexible ensemble framework (agnostic to base models) that delivers both strong predictive performance and asymptotically valid, powerful feature importance inference for regression.

发表机构

• University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）

机构由 AI 辅助整理，请以论文原文为准。