Logistic models are studied as a tool to convert output from numerical weather forecasting systems (deterministic and ensemble) into probability forecasts for binary events. A logistic model obtains by putting the logarithmic odds ratio equal to a linear combination of the inputs. As any statistical model, logistic models will suffer from over-fitting if the number of inputs is comparable to the number of forecast instances. Computational approaches to avoid over-fitting by regularisation are discussed, and efficient approaches for model assessment and selection are presented. A logit version of the so called lasso, which is originally a linear tool, is discussed. In lasso models, less important inputs are identified and discarded, thereby providing an efficient and automatic model reduction procedure. For this reason, lasso models are particularly appealing for diagnostic purposes.
We investigate the relationships between weather forecasting, weather derivatives, the Black-Scholes equation, Feynmann-Kac theory and the Fokker-Planck equation. There is one useful result, but on the whole the relations we present seem to be more interesting than practically useful.
We show that probabilistic weather forecasts of site specific temperatures can be dramatically improved by using seasonally varying rather than constant calibration parameters.
Historical observations of severe weather and simulated severe weather environments (i.e., features) from the Global Ensemble Forecast System v12 (GEFSv12) Reforecast Dataset (GEFS/R) are used in conjunction to train and test random forest (RF) machine learning (ML) models to probabilistically forecast severe weather out to days 4--8. RFs are trained with 9 years of the GEFS/R and severe weather reports to establish statistical relationships. Feature engineering is briefly explored to examine alternative methods for gathering features around observed events, including simplifying features usin…
Stockholm University（斯德哥尔摩大学）；Uppsala University（乌普萨拉大学）
Ensemble weather forecasts enable a measure of uncertainty to be attached to each forecast, by computing the ensemble's spread. However, generating an ensemble with a good spread-error relationship is far from trivial, and a wide range of approaches to achieve this have been explored -- chiefly in the context of numerical weather prediction models. Here, we aim to transform a deterministic neural network weather forecasting system into an ensemble forecasting system. We test four methods to generate the ensemble: random initial perturbations, retraining of the neural network, use of random dro…
The Met Office Space Weather Operations Centre was founded in 2014 and part of its remit is a daily Space Weather Technical Forecast to help the UK build resilience to space weather impacts; guidance includes four day geo-magnetic storm forecasts (GMSF) and X-ray flare forecasts (XRFF). It is crucial for forecasters, users, modelers and stakeholders to understand the strengths and weaknesses of these forecasts; therefore, it is important to verify against the most reliable truth data source available. The present study contains verification results for XRFFs using GOES-15 satellite data and GM…
In this paper, we present Pangu-Weather, a deep learning based system for fast and accurate global weather forecast. For this purpose, we establish a data-driven environment by downloading $43$ years of hourly global weather data from the 5th generation of ECMWF reanalysis (ERA5) data and train a few deep neural networks with about $256$ million parameters in total. The spatial resolution of forecast is $0.25^\circ\times0.25^\circ$, comparable to the ECMWF Integrated Forecast Systems (IFS). More importantly, for the first time, an AI-based method outperforms state-of-the-art numerical weather …
Computer-generated forecasts divide the earth's surface into gridboxes, each now ~25% of the size of London, and predict one value per gridbox. If weather varies markedly within a gridbox forecasts for specific sites inevitably fail. A completely new statistical post-processing method, using ensemble forecasts as input, anticipates two gridbox-weather-dependant factors: degree of variation in each gridbox, and bias on the gridbox scale. Globally, skill improves substantially; for extreme rainfall, for example, useful forecasts extend 5 days ahead. Without post-processing this limit is < 1 day.…
Imperial College London（伦敦帝国学院）；University of Exeter（埃克塞特大学）；Met Office（英国气象局）
The success of deep learning techniques over the last decades has opened up a new avenue of research for weather forecasting. Here, we take the novel approach of using a neural network to predict full probability density functions at each point in space and time rather than a single output value, thus producing a probabilistic weather forecast. This enables the calculation of both uncertainty and skill metrics for the neural network predictions, and overcomes the common difficulty of inferring uncertainty from these predictions. This approach is data-driven and the neural network is trained on…
University of Cambridge（剑桥大学）；Vector Institute（矢量研究所）；University of Toronto（多伦多大学）；Microsoft Research（微软研究院）；British Antarctic Survey（英国南极调查局）；The Alan Turing Institute（阿兰·图灵研究所）；European Centre for Medium-Range Weather Forecasts（欧洲中期天气预报中心）；Google Deepmind（谷歌DeepMind）
Rainfall is an important variable to be able to monitor and forecast across Africa, due to its impact on agriculture, food security, climate related diseases and public health. Numerical Weather Models (NWM) are an important component of this work, due to their complete spatial coverage, high resolution, and ability to forecast into the future. In this study, we seek to evaluate the spatiotemporal skill of short-term rainfall forecasts of rainfall across Africa. Specifically, the European Centre for Medium-Range Weather Forecasts (ECMWF) and the National Centers for Environmental Prediction-Gl…
We present a data-driven approach for forecasting global weather using graph neural networks. The system learns to step forward the current 3D atmospheric state by six hours, and multiple steps are chained together to produce skillful forecasts going out several days into the future. The underlying model is trained on reanalysis data from ERA5 or forecast data from GFS. Test performance on metrics such as Z500 (geopotential height) and T850 (temperature) improves upon previous data-driven approaches and is comparable to operational, full-resolution, physical models from GFS and ECMWF, at least…
Results of direct numerical simulations have been used to show that intensive thermal convection in a horizontal layer and on a hemisphere can be described by the distributed chaos approach. The vorticity and helicity dominated distributed chaos were considered for this purpose. Results of numerical simulations of the Weather Research and Forecast Model (with the moist convection and with the Coriolis effect) and of the Coupled Ocean-Atmosphere Mesoscale Prediction System (COAMPS) were also analysed to demonstrate applicability of this approach to the atmospheric processes. The ensemble foreca…
Cooperative Institute for Mesoscale Meteorological Studies（中尺度气象研究合作所）；NOAA/OAR/National Severe Storms Laboratory（美国国家海洋和大气管理局/海洋与大气研究/国家强风暴实验室）；School of Meteorology, University of Oklahoma（俄克拉荷马大学气象学院）；School of Computer Science, University of Oklahoma（俄克拉荷马大学计算机科学学院）
A primary goal of the National Oceanic and Atmospheric Administration (NOAA) Warn-on-Forecast (WoF) project is to provide rapidly updating probabilistic guidance to human forecasters for short-term (e.g., 0-3 h) severe weather forecasts. Maximizing the usefulness of probabilistic severe weather guidance from an ensemble of convection-allowing model forecasts requires calibration. In this study, we compare the skill of a simple method using updraft helicity against a series of machine learning (ML) algorithms for calibrating WoFS severe weather guidance. ML models are often used to calibrate se…
We assess the value of machine learning as an accelerator for the parameterisation schemes of operational weather forecasting systems, specifically the parameterisation of non-orographic gravity wave drag. Emulators of this scheme can be trained to produce stable and accurate results up to seasonal forecasting timescales. Generally, more complex networks produce more accurate emulators. By training on an increased complexity version of the existing parameterisation scheme we build emulators that produce more accurate forecasts. {For medium range forecasting we find evidence our emulators are m…
University of Hawaii Manoa（夏威夷大学马诺阿分校）；Nimbus AI LLC（Nimbus人工智能有限责任公司）
Forecasting future weather and climate is inherently difficult. Machine learning offers new approaches to increase the accuracy and computational efficiency of forecasts, but current methods are unable to accurately model uncertainty in high-dimensional predictions. Score-based diffusion models offer a new approach to modeling probability distributions over many dependent variables, and in this work, we demonstrate how they provide probabilistic forecasts of weather and climate variables at unprecedented resolution, speed, and accuracy. We apply the technique to day-ahead solar irradiance fore…
Extreme weather events have significant consequences, dominating the impact of climate on society. While high-resolution weather models can forecast many types of extreme events on synoptic timescales, long-term climatological risk assessment is an altogether different problem. A once-in-a-century event takes, on average, 100 years of simulation time to appear just once, far beyond the typical integration length of a weather forecast model. Therefore, this task is left to cheaper, but less accurate, low-resolution or statistical models. But there is untapped potential in weather model output: …
Intuitively, one would expect a more skillful forecast if predicting weather averaged over one week instead of the weather averaged over one day, and similarly for different spatial averaging areas. However, there are few systematic studies of averaging and forecast skill with modern forecasts, and it is therefore not clear how much improvement in forecast performance is produced via averaging. Here we present a direct investigation of averaging effects, based on data from operational numerical weather forecasts. Data is analyzed for precipitation and surface temperature, for lead times of rou…
Huracan: A skillful end-to-end data-driven system for ensemble data assimilation and weather prediction
Microsoft Corporation（微软公司）；Department of Earth System Science, Ministry of Education Key Laboratory for Earth System Modeling, Institute for Global Change Studies（教育部地球系统科学部门地球系统建模重点实验室，全球变化研究机构）；Tsinghua University（清华大学）
Numerical models outperform AI weather forecasts of record-breaking extremes
Research Institute for Statistics and Information Science, University of Geneva（统计与信息科学研究所，日内瓦大学）；Scientific Computing Center, Karlsruhe Institute of Technology（科学计算中心，卡尔斯鲁厄理工大学）；Institute for Atmospheric and Climate Science, Department of Environmental Systems Science, ETH Zurich（大气与气候科学研究所，环境系统科学系，苏黎世联邦理工学院）；Department of Compound Environmental Risks,Helmholtz Centre for Environmental Research – UFZ（复合环境风险部门，环境研究中心——UFZ）；Department of Hydro Sciences, TUD Dresden University of Technology（水科学系，德累斯顿理工大学）
Can we create a `race to the top' for weather forecasts to inform smallholder farmer decisions?
University of Chicago（芝加哥大学）；Columbia University（哥伦比亚大学）；University of Witwatersrand（威特沃特斯兰德大学）；AfriClimate AI（非洲气候人工智能）；University of Leeds（利兹大学）；University of Oxford（牛津大学）；African Centre of Meteorological Applications for Development（非洲气象应用发展中心）；NORCE Research AS（挪威NORCE研究所）
FengWu-GHR: Learning the Kilometer-scale Medium-range Global Weather Forecasting
Shanghai AI Laboratory（上海人工智能实验室）；Hong Kong University of Science and Technology（香港科技大学）；Nanjing University of Information Science and Technology（南京信息工程大学）；National Meteorological Information Center of CMA（中国气象局国家气象信息中心）；National Meteorological Center of CMA（中国气象局国家气象中心）
FuXi-2.0: Advancing machine learning weather forecasting model for practical applications
Artificial Intelligence Innovation and Incubation Institute, Fudan University（复旦大学人工智能创新与孵化研究院）；Shanghai Academy of Artificial Intelligence for Science（上海科学人工智能研究院）
Global Forecasting of Tropical Cyclone Intensity Using Neural Weather Models
Faculty of Geosciences and Environment, University of Lausanne（洛桑大学地球与环境科学学院）；Expertise Center for Climate Extremes, University of Lausanne（洛桑大学气候极端研究中心）；Environmental Remote Sensing Laboratory, EPFL（洛桑联邦理工学院环境遥感实验室）
We discuss an approach to probabilistic forecasting based on two chained machine-learning steps: a dimensional reduction step that learns a reduction map of predictor information to a low-dimensional space in a manner designed to preserve information about forecast quantities; and a density estimation step that uses the probabilistic machine learning technique of normalizing flows to compute the joint probability density of reduced predictors and forecast quantities. This joint density is then renormalized to produce the conditional forecast distribution. In this method, probabilistic calibrat…
