特征袋装提供稳定性

中文摘要 AI 辅助

我们从算法稳定性的角度研究特征袋装。特征袋装是一种集成策略，它聚合在随机子采样特征子集上训练的基学习器，可能以数据依赖的方式进行。我们引入特征不稳定性（FI），它是实例不稳定性（II）在特征轴上的对应概念，用于衡量移除单个特征的敏感性。II或FI的值越小，对应稳定性越强，我们的实验表明FI捕获的与泛化相关的信息与II互补。在该框架内，我们分析了参数线性模型和受随机森林中递归特征子采样启发的无模型设置下的特征袋装，在两种设置中，我们均建立了形式化保证，表明特征袋装相对于其非袋装对应物可提升相关稳定性，且在更激进的子采样下提升幅度更大。我们进一步表明，适度的袋装轮次足以接近无限袋装的稳定性水平。

英文摘要

We study feature bagging through the lens of algorithmic stability. Feature bagging is an ensemble strategy that aggregates base learners trained on randomly subsampled feature subsets, possibly in a data-dependent manner. We introduce feature instability (FI), the feature-axis analogue of instance instability (II), which measures sensitivity to removing a single feature. Smaller values of II or FI correspond to stronger stability, and our experiments show that FI captures generalization-relevant information complementary to II. Within this framework, we analyze feature bagging in both a parametric linear model and a model-free setting inspired by recursive feature subsampling in random forests. In both settings, we establish formal guarantees showing that feature bagging improves the relevant stability relative to its non-bagged counterpart, with larger improvements under more aggressive subsampling. We further show that a modest number of bagging rounds is sufficient to approach the infinite-bagging stability level.

发表机构

• School of Statistics, East China Normal University（华东师范大学统计学院）
• University of Toronto（多伦多大学）

机构由 AI 辅助整理，请以论文原文为准。