Use of the likelihood for measuring the skill of probabilistic forecasts
Stephen Jewson
备注：
Version 1 (3rd August 2003) contained some incorrect statements about the relationship between the likelihood and the Brier score. These have now been removed
We define the likelihood and give a number of justifications for its use as a skill measure for probabilistic forecasts. We describe a number of different scores based on the likelihood, and briefly investigate the relationships between the likelihood, the mean square error and the ignorance.
We quantify the value of sub-seasonal forecasts for a real-world prediction problem: the forecasting of French month-ahead energy demand. Using surface temperature as a predictor, we construct a trading strategy and assess the financial value of using meteorological forecasts, based on actual energy demand and price data. We show that forecasts with lead times greater than 2 weeks can have value for this application, both on their own and in conjunction with shorter range forecasts, especially during boreal winter. We consider a cost/loss framework based on this example, and show that while it…
We analyse Indian summer monsoon (ISM) seasonal reforecasts by CFSv2 model, initiated from January (4-month lead time, L4) through May (0-month lead time, L0) initial conditions (ICs), to examine the cause for highest all-India ISM rainfall (ISMR) forecast skill with February (L3) ICs. The reported highest L3 skill is based on correlation between observed and predicted interannual variation (IAV) of ISMR. Other scores such as mean error, bias, RMSE, mean, standard deviation and coefficient of variation, indicate higher or comparable skill for April(L1)/May(L0) ICs. Though theory suggests that …
Historical observations of severe weather and simulated severe weather environments (i.e., features) from the Global Ensemble Forecast System v12 (GEFSv12) Reforecast Dataset (GEFS/R) are used in conjunction to train and test random forest (RF) machine learning (ML) models to probabilistically forecast severe weather out to days 4--8. RFs are trained with 9 years of the GEFS/R and severe weather reports to establish statistical relationships. Feature engineering is briefly explored to examine alternative methods for gathering features around observed events, including simplifying features usin…
We assess the skill and reliability of forecasts of winter and summer temperature, wind speed and irradiance over China, using the GloSea5 seasonal forecast system. Skill in such forecasts is important for the future development of seasonal climate services for the energy sector, allowing better estimates of forthcoming demand and renewable electricity supply. We find that although overall the skill from the direct model output is patchy, some high-skill regions of interest to the energy sector can be identified. In particular, winter mean wind speed is skilfully forecast around the coast of t…
All numerical weather prediction models used for the wind industry need to produce their forecasts starting from the main synoptic hours 00, 06, 12, and 18 UTC, once the analysis becomes available. The six-hour latency time between two consecutive model runs calls for strategies to fill the gap by providing new accurate predictions having, at least, hourly frequency. This is done to accommodate the request of frequent, accurate and fresh information from traders and system regulators to continuously adapt their work strategies. Here, we propose a strategy where quasi-real time observed wind sp…
Intuitively, one would expect a more skillful forecast if predicting weather averaged over one week instead of the weather averaged over one day, and similarly for different spatial averaging areas. However, there are few systematic studies of averaging and forecast skill with modern forecasts, and it is therefore not clear how much improvement in forecast performance is produced via averaging. Here we present a direct investigation of averaging effects, based on data from operational numerical weather forecasts. Data is analyzed for precipitation and surface temperature, for lead times of rou…
Predictive skill of complex models is often not uniform in model-state space; in weather forecasting models, for example, the skill of the model can be greater in populated regions of interest than in "remote" regions of the globe. Given a collection of models, a multi-model forecast system using the cross pollination in time approach can be generalised to take advantage of instances where some models produce systematically more accurate forecast of some components of the model-state. This generalisation is stated and then successfully demonstrated in a moderate ~40 dimensional nonlinear dynam…
Accumulated precipitation forecasts are of high socioeconomic importance for agriculturally dominated societies in northern tropical Africa. In this study, we analyze the performance of nine operational global ensemble prediction systems (EPSs) relative to climatology-based forecasts for 1 to 5-day accumulated precipitation based on the monsoon seasons 2007-2014 for three regions within northern tropical Africa. To assess the full potential of raw ensemble forecasts across spatial scales, we apply state-of-the-art statistical postprocessing methods in form of Bayesian Model Averaging (BMA) and…
Is the 2026 East Asia Summer Extreme Heat Forecast Credible?
Nanjing University of Information Science and Technology（南京信息工程大学）；CAMS（中国气象局）；Center for Earth System Modeling and Prediction of CMA (CEMC)（中国气象局地球系统模式与预测中心）；China Meteorological Administration（中国气象局）
The seamless forecast approach of subseasonal to seasonal scale variability has been succeeding in the forecast of multiple meteorological scales in a uniform framework. In this paradigm, it is hypothesized that reduction in initial error in dynamical forecast would help to reduce forecast error in extended lead-time up to 2-3 weeks. This is tested in a version of operational extended range forecasts based on Climate Forecast System version 2 (CFSv2) developed at Indian Institute of Tropical Meteorology (IITM), Pune. Forecast skills are assessed to understand the role of initial errors on the …
Global Evaluation of AI and NWP Precipitation Forecasts During Atmospheric River Events
University of Oklahoma（俄克拉荷马大学）；NSF AI Institute for Research on Trustworthy AI in Weather, Climate, and Coastal Oceanography（美国国家科学基金会天气、气候与沿海海洋学可信人工智能研究人工智能研究所）；Brightband
UC Berkeley（加州大学伯克利分校）；Lawrence Berkeley National Laboratory（劳伦斯伯克利国家实验室）
Forecasting global precipitation patterns and, in particular, extreme precipitation events is of critical importance to preparing for and adapting to climate change. Making accurate high-resolution precipitation forecasts using traditional physical models remains a major challenge in operational weather forecasting as they incur substantial computational costs and struggle to achieve sufficient forecast skill. Recently, deep-learning-based models have shown great promise in closing the gap with numerical weather prediction (NWP) models in terms of precipitation forecast skill, opening up excit…
Advancing Subseasonal Forecasting with Machine Learning
Harvard College（哈佛学院）；University of Toronto（多伦多大学）；Instituto de Matemática Pura e Aplicada（数学与应用数学研究所）；Massachusetts Institute of Technology（麻省理工学院）；Atmospheric and Environmental Research（大气与环境研究）；Microsoft Corporation（微软公司）；Rhiza Research（Rhiza研究）；Microsoft Research New England（微软新英格兰研究院）；European Centre for Medium-Range Weather Forecasts（欧洲中期天气预报中心）
In-situ ocean wave observations are critical to improve model skill and validate remote sensing wave measurements. Historically, such observations are extremely sparse due to the large costs and complexity of traditional wave buoys and sensors. In this work, we present a recently deployed network of free-drifting satellite-connected surface weather buoys that provide long-dwell coverage of surface weather in the northern Pacific Ocean basin. To evaluate the leading-order improvements to model forecast skill using this distributed sensor network, we implement a widely-used data assimilation tec…
On the Predictive Skill of Artificial Intelligence-based Weather Models for Extreme Events using Uncertainty Quantification
Applied Machine Learning Group（应用机器学习组）；Fraunhofer Heinrich-Hertz Institute（弗劳恩霍夫海因里希-赫兹研究所）；Signal Theory and Communications Dept.（信号理论与通信系）；Universidad Carlos III de Madrid (UC3M)（马德里卡洛斯三世大学）
Uncertainty Quantification in Forecast Comparisons
University of Wuppertal（伍珀塔尔大学）；Heidelberg Institute for Theoretical Studies（海德堡理论研究所）；Goethe University Frankfurt, Faculty of Economics and Business（法兰克福歌德大学经济与商业学院）；Marburg University, Department of Mathematics and Computer Science（马尔堡大学数学与计算机系）
