arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2302.13511stat.MEstat.ML
收藏

面向随机集成模型的外推交叉验证法

Extrapolated cross-validation for randomized ensembles

Jin-Hong Du, Pratik Patil, Kathryn Roeder, Arun Kumar Kuchibhotla

AI总结:

针对随机集成模型参数调优效率低的问题,提出基于袋外误差与风险外推的ECV交叉验证法,可在高维场景下实现δ-最优调参,兼具更高准确率与更低计算成本。

摘要:

从金融学到基因组学,bagging(装袋法)、随机森林等集成方法在各领域应用十分广泛。尽管这类方法十分普及,但关于集成参数高效调优的问题却相对较少受到关注。本文提出一种交叉验证方法ECV(Extrapolated Cross-Validation,外推交叉验证),用于调优随机集成模型的集成规模与子样本规模。\n我们的方法基于两个核心要素:一是利用袋外误差得到的小规模集成初始估计量,二是借助预测风险分解结构提出的新型风险外推技术。通过证明该风险外推技术在集成规模与子样本规模上的一致收敛性,我们表明在平方预测风险指标下,ECV可得到相对于最优调参风险的δ-最优集成模型。我们的理论适用于通用集成预测器,仅需较弱的矩假设,且支持特征维度随样本量增长的高维场景。\n作为实际案例研究,我们使用ECV结合随机森林,在单细胞多组学任务中从基因表达数据预测表面蛋白丰度。与样本拆分交叉验证和K折交叉验证相比,ECV无需拆分样本即可实现更高的准确率,同时由于采用了风险外推技术,其计算成本显著更低。额外的数值实验结果验证了在最大集成规模受计算资源约束的条件下,ECV针对多种常见集成预测器的有限样本准确率。

英文摘要:

Ensemble methods such as bagging and random forests are ubiquitous in various fields, from finance to genomics. Despite their prevalence, the question of the efficient tuning of ensemble parameters has received relatively little attention. This paper introduces a cross-validation method, ECV (Extrapolated Cross-Validation), for tuning the ensemble and subsample sizes in randomized ensembles. Our method builds on two primary ingredients: initial estimators for small ensemble sizes using out-of-bag errors and a novel risk extrapolation technique that leverages the structure of prediction risk decomposition. By establishing uniform consistency of our risk extrapolation technique over ensemble and subsample sizes, we show that ECV yields $δ$-optimal (with respect to the oracle-tuned risk) ensembles for squared prediction risk. Our theory accommodates general ensemble predictors, only requires mild moment assumptions, and allows for high-dimensional regimes where the feature dimension grows with the sample size. As a practical case study, we employ ECV to predict surface protein abundances from gene expressions in single-cell multiomics using random forests. In comparison to sample-split cross-validation and $K$-fold cross-validation, ECV achieves higher accuracy avoiding sample splitting. At the same time, its computational cost is considerably lower owing to the use of the risk extrapolation technique. Additional numerical results validate the finite-sample accuracy of ECV for several common ensemble predictors under a computational constraint on the maximum ensemble size.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑