arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2111.08478cs.LG
收藏

空间机器学习模型诊断:一种模型无关的基于距离的方法

Spatial machine-learning model diagnostics: a model-agnostic distance-based approach

Friedrich Schiller University Jena(耶拿弗里德里希·席勒大学)

Alexander Brenning

AI总结:

本文提出模型无关的SPEPs与SVIPs,基于预测距离诊断空间ML模型的预测技能和变量重要性,并通过环境区域化与遥感土地覆盖分类案例验证其价值。

摘要:

尽管在解释黑盒机器学习(ML)模型方面已取得显著进展,但仍明显缺乏能够从预测技能和变量重要性角度阐明ML模型空间行为的诊断工具。本研究提出空间预测误差剖面(SPEPs)和空间变量重要性剖面(SVIPs),作为面向空间预测模型的新型模型无关评估与解释工具,其重点是预测距离。通过两个案例研究证明了它们的适用性:一个代表环境科学背景下的区域化任务,另一个是来自遥感土地覆盖分类的分类任务。在这些案例研究中,地质统计方法、线性模型、random forest(随机森林)和混合算法的SPEPs与SVIPs表现出显著差异,但也存在相关的相似之处。文中概述了相关交叉验证技术的局限性,并提出建模者应将模型评估和解释的重点放在预期的空间预测范围上。相比之下,自相关范围并不是定义空间交叉验证测试集的合适标准。这些新型诊断工具丰富了空间数据科学的工具箱,并可能改进ML模型的解释、选择和设计。

英文摘要:

While significant progress has been made towards explaining black-box machine-learning (ML) models, there is still a distinct lack of diagnostic tools that elucidate the spatial behaviour of ML models in terms of predictive skill and variable importance. This contribution proposes spatial prediction error profiles (SPEPs) and spatial variable importance profiles (SVIPs) as novel model-agnostic assessment and interpretation tools for spatial prediction models with a focus on prediction distance. Their suitability is demonstrated in two case studies representing a regionalization task in an environmental-science context, and a classification task from remotely-sensed land cover classification. In these case studies, the SPEPs and SVIPs of geostatistical methods, linear models, random forest, and hybrid algorithms show striking differences but also relevant similarities. Limitations of related cross-validation techniques are outlined, and the case is made that modelers should focus their model assessment and interpretation on the intended spatial prediction horizon. The range of autocorrelation, in contrast, is not a suitable criterion for defining spatial cross-validation test sets. The novel diagnostic tools enrich the toolkit of spatial data science, and may improve ML model interpretation, selection, and design.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑