arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2203.04552stat.ME
收藏

模型选择的交叉验证:生态学实例入门

Cross validation for model selection: a primer with examples from ecology

Luke Yates, Zach Aandahl, Shane A. Richards, Barry W. Brook

摘要:

生态学中越来越多地使用模型选择原则进行统计推断,其基础是信息准则(IC)和交叉验证(CV)技术。虽然IC技术,如Akaike的信息准则,在生态学中历史上更受欢迎,CV是一个多功能的,越来越多地使用的替代品。CV使用数据分割来基于(样本外)预测性能估计模型得分,即使在无法导出可能性(例如,机器学习)或精确地计数参数(例如,混合效应模型和惩罚回归)。在这里,我们提供了一个入门的理解和应用CV在生态学。我们回顾了CV的常用变体,包括近似方法,并根据统计背景对其使用提出建议。我们解释了一些重要的-但经常被忽视-CV的技术方面,如偏差校正,估计不确定性,分数选择和简约的选择规则。我们还解决误解(和真理)的使用CV的障碍,包括计算成本和易于实施,并澄清CV和信息理论方法之间的关系,模型选择。本文包括两个生态案例研究,说明了技术的应用。我们的结论是,CV为基础的模型选择应广泛应用于生态分析,因为它的稳健估计属性和广泛的情况下,它是适用的。特别是,我们建议使用留一法(LOO)或近似LOO CV来最小化偏倚,或者如果K<10,则使用偏倚校正的K倍CV。为了减轻过度拟合,我们建议通过修改后的一个标准误差规则,占过度拟合的主要原因:得分估计的不确定性校准选择。

英文摘要:

The growing use of model-selection principles in ecology for statistical inference is underpinned by information criteria (IC) and cross-validation (CV) techniques. Although IC techniques, such as Akaike's Information Criterion, have been historically more popular in ecology, CV is a versatile and increasingly used alternative. CV uses data splitting to estimate model scores based on (out-of-sample) predictive performance, which can be used even when it is not possible to derive a likelihood (e.g., machine learning) or count parameters precisely (e.g., mixed-effects models and penalised regression). Here we provide a primer to understanding and applying CV in ecology. We review commonly applied variants of CV, including approximate methods, and make recommendations for their use based on the statistical context. We explain some important -- but often overlooked -- technical aspects of CV, such as bias correction, estimation uncertainty, score selection, and parsimonious selection rules. We also address misconceptions (and truths) about impediments to the use of CV, including computational cost and ease of implementation, and clarify the relationship between CV and information-theoretic approaches to model selection. The paper includes two ecological case studies which illustrate the application of the techniques. We conclude that CV-based model selection should be widely applied to ecological analyses, because of its robust estimation properties and the broad range of situations for which it is applicable. In particular, we recommend using leave-one-out (LOO) or approximate LOO CV to minimise bias, or otherwise K-fold CV using bias correction if K<10. To mitigate overfitting, we recommend calibrated selection via the modified one-standard-error rule which accounts for the predominant cause of overfitting: score-estimation uncertainty.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑