While significant progress has been made towards explaining black-box machine-learning (ML) models, there is still a distinct lack of diagnostic tools that elucidate the spatial behaviour of ML models in terms of predictive skill and variable importance. This contribution proposes spatial prediction error profiles (SPEPs) and spatial variable importance profiles (SVIPs) as novel model-agnostic assessment and interpretation tools for spatial prediction models with a focus on prediction distance. Their suitability is demonstrated in two case studies representing a regionalization task in an environmental-science context, and a classification task from remotely-sensed land cover classification. In these case studies, the SPEPs and SVIPs of geostatistical methods, linear models, random forest, and hybrid algorithms show striking differences but also relevant similarities. Limitations of related cross-validation techniques are outlined, and the case is made that modelers should focus their model assessment and interpretation on the intended spatial prediction horizon. The range of autocorrelation, in contrast, is not a suitable criterion for defining spatial cross-validation test sets. The novel diagnostic tools enrich the toolkit of spatial data science, and may improve ML model interpretation, selection, and design.
SpatioTemporal Causal Network Diagnostics for Geographic Tipping Point Early Warning
Jiangsu Center for Collaborative Innovation in Geographical Information Resource Development and Application(江苏省地理信息资源开发与应用协同创新中心);National Center for Applied Mathematics, Tianjin University(天津大学国家应用数学中心)
Evaluating Black-Box Vulnerabilities with Wasserstein-Constrained Data Perturbations
INRIA Regalia & ANITI Institut de Mathématiques de Toulouse, France Centre INRIA de l’Université de Bordeaux(INRIA里加大学与ANITI数学研究所,法国图卢兹数学研究所,INRIA波尔多中心)
Towards Coordinate- and Dimension-Agnostic Machine Learning for Partial Differential Equations
Department of Chemical and Biomolecular Engineering(化学与生物分子工程系);Johns Hopkins University(约翰·霍普金斯大学);Department of Applied Mathematics and Statistics(应用数学与统计学系);Microsoft Azure Quantum(微软Azure量子)
Diagnostic Spatio-temporal Transformer with Faithful Encoding
Ikerlan Technology Research Centre(伊克兰技术研究中心);Basque Research and Technology Alliance (BRTA)(巴斯克研究与技术联盟(BRTA));Mondragon University(蒙德拉贡大学);IBM Research(IBM研究院)