Verifying probabilistic forecasts for extreme events is a highly active research area because popular media and public opinions are naturally focused on extreme events, and biased conclusions are readily made. In this context, classical verification methods tailored for extreme events, such as thresholded and weighted scoring rules, have undesirable properties that cannot be mitigated, and the well-known continuous ranked probability score (CRPS) is no exception.
In this paper, we define a formal framework for assessing the behavior of forecast evaluation procedures with respect to extreme events, which we use to demonstrate that assessment based on the expectation of a proper score is not suitable for extremes. Alternatively, we propose studying the properties of the CRPS as a random variable by using extreme value theory to address extreme event verification. An index is introduced to compare calibrated forecasts, which summarizes the ability of probabilistic forecasts for predicting extremes. The strengths and limitations of this method are discussed using both theoretical arguments and simulations.
Distributional Regression U-Nets for the Postprocessing of Precipitation Ensemble Forecasts
Université de Franche Comté（弗朗什-孔泰大学）；CNRS（法国国家科学研究中心）；Laboratoire des Sciences du Climat et de l’Environnement（气候与环境科学实验室）；CEA（法国原子能和替代能源委员会）；UVSQ（凡尔赛大学）；IPSL（巴黎地球物理研究所）；CNRM（法国国家气象研究中心）；Université de Toulouse（图卢兹大学）；Météo-France（法国气象局）
Typically, point forecasting methods are compared and assessed by means of an error measure or scoring function, such as the absolute error or the squared error. The individual scores are then averaged over forecast cases, to result in a summary measure of the predictive performance, such as the mean absolute error or the (root) mean squared error. I demonstrate that this common practice can lead to grossly misguided inferences, unless the scoring function and the forecasting task are carefully matched. Effective point forecasting requires that the scoring function be specified ex ante, or tha…
