The recent developments in the machine learning domain have enabled the development of complex multivariate probabilistic forecasting models. Therefore, it is pivotal to have a precise evaluation method to gauge the performance and predictability power of these complex methods. To do so, several evaluation metrics have been proposed in the past (such as Energy Score, Dawid-Sebastiani score, variogram score), however, they cannot reliably measure the performance of a probabilistic forecaster. Recently, CRPS-sum has gained a lot of prominence as a reliable metric for multivariate probabilistic forecasting. This paper presents a systematic evaluation of CRPS-sum to understand its discrimination ability. We show that the statistical properties of target data affect the discrimination ability of CRPS-Sum. Furthermore, we highlight that CRPS-Sum calculation overlooks the performance of the model on each dimension. These flaws can lead us to an incorrect assessment of model performance. Finally, with experiments on the real-world dataset, we demonstrate that the shortcomings of CRPS-Sum provide a misleading indication of the probabilistic forecasting performance method. We show that it is easily possible to have a better CRPS-Sum for a dummy model, which looks like random noise, in comparison to the state-of-the-art method.
Fixing the Pitfalls of Probabilistic Time-Series Forecasting Evaluation by Kernel Quadrature
Lattice Lab, Toyota Motor Corporation / Machine Learning Research Group, University of Oxford（丰田公司 lattice 实验室 / 机器学习研究组，牛津大学）；The University of Osaka / Lattice Lab, Toyota Motor Corporation / RIKEN AIP（大阪大学 / 丰田公司 lattice 实验室 / RIKEN AIP）；Machine Learning Research Group, University of Oxford（机器学习研究组，牛津大学）
FuXi-ENS: A machine learning model for medium-range ensemble weather forecasting
Artificial Intelligence Innovation and Incubation Institute, Fudan University（复旦大学人工智能创新与孵化研究院）；Shanghai Academy of Artificial Intelligence for Science（上海科学人工智能研究院）；National Meteorological Information Center, China Meteorological Administration（中国气象局国家气象信息中心）
PROSWIN: Probabilistic Solar Wind Speed Forecasting Using Deep Distributional Regression From Solar Images
GFZ Helmholtz Centre for Geosciences（亥姆霍兹波茨坦地学研究中心）；Technical University of Berlin（柏林工业大学）；University of Potsdam（波茨坦大学）；University of California Los Angeles（加利福尼亚大学洛杉矶分校）；German Aerospace Center（德国航空航天中心）；Columbia University（哥伦比亚大学）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；Einstein Center Digital Future（爱因斯坦数字未来中心）
Single-stage joint diffusion of sky images and clear-sky index for ultra-short-term irradiance forecasting: what the frame-to-irradiance stage discards
Tyan-WP: A Wind Power Foundation Model for Ultra-Short-Term Probabilistic Forecasting
School of Information Science and Technology, University of Science and Technology of China（中国科学技术大学信息科学技术学院）；China Electric Power Research Institute（中国电力科学研究院）
SPECTRA: State-Space Exogenous Context and Temporal-Frequency Resolution Architecture for Probabilistic Energy Forecasting
Shanghai Advanced Research Institute, Chinese Academy of Sciences（中国科学院上海先进研究院）；University of Chinese Academy of Sciences（中国科学院大学）；School of Physical Science and Technology, Shanghaitech University（上海科技大学物理科学与技术学院）；State Power Investment Corporation Limited（国家电力投资集团有限公司）；China University of Petroleum（中国石油大学）
Characterization and forecasting of national-scale solar power ramp events
School of Engineering and Computer Science, Bern University of Applied Sciences（伯尔尼应用科学大学工程与计算机科学学院）；Department of Geosciences and Remote Sensing, TU Delft（代尔夫特理工大学地球科学与遥感系）
Uncertainty-Aware End-to-End AI Weather Forecasting: Disentangling Observation and Model Contributions
Fraunhofer Heinrich-Hertz Institute（弗劳恩霍夫海因里希·赫兹研究所）；Technische Universität Berlin（柏林工业大学）；BIFOLD - Berlin Institute for the Foundations of Learning and Data（柏林学习与数据基础研究所（BIFOLD））
Diffusion Precipitation Generation with Medium-range Weather Forecasting
Tsinghua University（清华大学）；The Chinese University of Hong Kong（香港中文大学）；The Hong Kong University of S & T（香港科技大学）；Sun Yat-Sen University（中山大学）；NSCC-SZ（国家超级计算深圳中心）；China Meteorological Administration（中国气象局）；GBA Weather Research Center（粤港澳大湾区天气研究中心）；Huawei Technologies Co., Ltd（华为技术有限公司）
Zero and Few Shot Load Forecasting with Large Language Models
School of Electrical Engineering, Southeast University（东南大学电气工程学院）；Wind Engineering and Renewable Energy Laboratory, Ecole Polytechnique Federale de Lausanne (EPFL)（瑞士联邦理工学院洛桑分校风能与可再生能源实验室）；College of Electrical Engineering and New Energy, China Three Gorges University（中国三峡大学电气工程与新能源学院）；Department of Electrical and Electronic Engineering, Imperial College London（伦敦帝国理工学院电子与电气工程系）；The Department of Automation, School of Automation and Intelligent Sensing, Shanghai Jiao Tong University（上海交通大学自动化与智能感知学院）；The Key Laboratory of System Control and Information Processing, Ministry of Education of China, Shanghai（中国教育部系统控制与信息处理重点实验室，上海）；State Key Laboratory of Submarine Geoscience, Shanghai（上海 submarine 地球科学国家重点实验室）；Institute of Energy Systems, Energy Efficiency and Energy Economic, TU Dortmund University（德意志图林根大学能源系统、能效与能源经济研究所）
Microsoft（微软）；Department of Atmospheric Sciences, University of Washington（华盛顿大学大气科学系）；Microsoft Research（微软研究院）
We present an ensemble prediction system using a Deep Learning Weather Prediction (DLWP) model that recursively predicts key atmospheric variables with six-hour time resolution. This model uses convolutional neural networks (CNNs) on a cubed sphere grid to produce global forecasts. The approach is computationally efficient, requiring just three minutes on a single GPU to produce a 320-member set of six-week forecasts at 1.4° resolution. Ensemble spread is primarily produced by randomizing the CNN training process to create a set of 32 DLWP models with slightly different learned weights. Althou…
Royal Netherlands Meteorological Institute (KNMI)（荷兰皇家气象研究所（KNMI））；Utrecht University（乌得勒支大学）
Current statistical post-processing methods for probabilistic weather forecasting are not capable of using full spatial patterns from the numerical weather prediction (NWP) model. In this paper we incorporate spatial wind speed information by using convolutional neural networks (CNNs) and obtain probabilistic wind speed forecasts in the Netherlands for 48 hours ahead, based on KNMI's deterministic Harmonie-Arome NWP model. The probabilistic forecasts from the CNNs are shown to have higher Brier skill scores for medium to higher wind speeds, as well as a better continuous ranked probability sco…
Ensemble Graph Neural Networks for Probabilistic Sea Surface Temperature Forecasting via Input Perturbations
Centro de Tecnologías de la Imagen Instituto Universitario de Cibernética, Empresas y Sociedad (IUCES) University of Las Palmas de Gran Canaria, Spain（图像技术中心大学计算机科学、企业和社会研究院（IUCES）大学拉斯帕尔马斯德Gran Canaria分校）
DiffSTG: Probabilistic Spatio-Temporal Graph Forecasting with Denoising Diffusion Models
School of Computer and Information Technology, Beijing Jiaotong University（北京交通大学计算机与信息技术学院）；National University of Singapore（新加坡国立大学）；DAMO Academy, Alibaba Group（阿里巴巴集团达摩院）；Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
Denoising diffusion probabilistic models are a promising new class of generative models that mark a milestone in high-quality image generation. This paper showcases their ability to sequentially generate video, surpassing prior methods in perceptual and probabilistic forecasting metrics. We propose an autoregressive, end-to-end optimized video diffusion model inspired by recent advances in neural video compression. The model successively generates future frames by correcting a deterministic next-frame prediction using a stochastic residual generated by an inverse diffusion process. We compare …
EMFusion: Uncertainty-Aware Conditional Diffusion Model for Multivariate Narrow-band Exposure Forecasting
department of Electrical Engineering and Computer Science, York University（电气工程与计算机科学系，约克大学）；School of Electrical and Electronic Engineering, Huazhong University of Science and Technology（电子与电气工程学院，华中科技大学）；Central China Branch of State Grid Corporation of China（国家电网公司中部分部）；Department of Electronic Engineering, University of Rome Tor Vergata（罗马大学Tor Vergata电子工程系）；Consorzio Nazionale Interuniversitario per le Telecomunicazioni (CNIT)（国家大学间电信研究会（CNIT））
