Forecasting the formation and development of clouds is a central element of modern weather forecasting systems. Incorrect clouds forecasts can lead to major uncertainty in the overall accuracy of weather forecasts due to their intrinsic role in the Earth's climate system. Few studies have tackled this challenging problem from a machine learning point-of-view due to a shortage of high-resolution datasets with many historical observations globally. In this paper, we present a novel satellite-based dataset called ``CloudCast''. It consists of 70,080 images with 10 different cloud types for multiple layers of the atmosphere annotated on a pixel level. The spatial resolution of the dataset is 928 x 1530 pixels (3x3 km per pixel) with 15-min intervals between frames for the period 2017-01-01 to 2018-12-31. All frames are centered and projected over Europe. To supplement the dataset, we conduct an evaluation study with current state-of-the-art video prediction methods such as convolutional long short-term memory networks, generative adversarial networks, and optical flow-based extrapolation methods. As the evaluation of video prediction is difficult in practice, we aim for a thorough evaluation in the spatial and temporal domain. Our benchmark models show promising results but with ample room for improvement. This is the first publicly available global-scale dataset with high-resolution cloud types on a high temporal granularity to the authors' best knowledge.
University of Oxford（牛津大学）；Inria（法国国家信息与自动化研究所）；University College London（伦敦大学学院）；University of Warsaw（华沙大学）
Extreme precipitation events, such as violent rainfall and hail storms, routinely ravage economies and livelihoods around the developing world. Climate change further aggravates this issue. Data-driven deep learning approaches could widen the access to accurate multi-day forecasts, to mitigate against such events. However, there is currently no benchmark dataset dedicated to the study of global precipitation forecasts. In this paper, we introduce \textbf{RainBench}, a new multi-modal benchmark dataset for data-driven precipitation forecasting. It includes simulated satellite data, a selection …
California Crop Yield Benchmark: Combining Satellite Image, Climate, Evapotranspiration, and Soil Data Layers for County-Level Yield Forecasting of Over 70 Crops
University of California, Davis（加州大学戴维斯分校）；Texas A&M University-Corpus Christi（德克萨斯A&M大学-科斯蒂西）；AI Institute for Food Systems（人工智能食品系统研究所）
Multi-modal learning for geospatial vegetation forecasting
Max-Planck-Institute for Biogeochemistry（马克斯·普朗克生物地球化学研究所）；ELLIS Unit Jena（耶拿ELLIS单元）；ETH Zürich（苏黎世联邦理工学院）；Hong Kong University of Science and Technology（香港科技大学）
Max-Planck-Institute for Biogeochemistry（马克斯·普朗克生物地球化学研究所）；German Aerospace Center (DLR)（德国航空航天中心）；University of Jena（耶拿大学）；Michael-Stifel-Center Jena for Data-driven and Simulation Science（耶拿米夏埃尔·施蒂费尔数据驱动与仿真科学中心）；Technische Universität Berlin（柏林工业大学）
Satellite images are snapshots of the Earth surface. We propose to forecast them. We frame Earth surface forecasting as the task of predicting satellite imagery conditioned on future weather. EarthNet2021 is a large dataset suitable for training deep neural networks on the task. It contains Sentinel 2 satellite imagery at 20m resolution, matching topography and mesoscale (1.28km) meteorological variables packaged into 32000 samples. Additionally we frame EarthNet2021 as a challenge allowing for model intercomparison. Resulting forecasts will greatly improve (>x50) over the spatial resolution f…
Carnegie Mellon University（卡内基梅隆大学）；IBM Research Labs（IBM研究院实验室）；GE Power（通用电气电力公司）
Ahead-of-time forecasting of incident solar-irradiance on a panel is indicative of expected energy yield and is essential for efficient grid distribution and planning. Traditionally, these forecasts are based on meteorological physics models whose parameters are tuned by coarse-grained radiometric tiles sensed from geo-satellites. This research presents a novel application of deep neural network approach to observe and estimate short-term weather effects from videos. Specifically, we use time-lapsed videos (sky-videos) obtained from upward facing wide-lensed cameras (sky-cameras) to directly e…
Spatio-Temporal Wildfire Spread Prediction in Canada using a Video Swin-Hybrid-U-Net and Satellite Imagery
Faculty of Science, University of Alberta（阿尔伯塔大学理学院）；Interdisciplinary Lab for Mathematical Ecology & Epidemiology (ILMEE), University of Alberta（阿尔伯塔大学数学生态与流行病学跨学科实验室）；Department of Mathematical and Statistical Sciences, University of Alberta（阿尔伯塔大学数学与统计科学系）
Max-Planck-Institute for Biogeochemistry（马克斯·普朗克生物地球化学研究所）；University of Jena（耶拿大学）；German Aerospace Center (DLR)（德国航空航天中心）
Climate change is global, yet its concrete impacts can strongly vary between different locations in the same region. Seasonal weather forecasts currently operate at the mesoscale (> 1 km). For more targeted mitigation and adaptation, modelling impacts to < 100 m is needed. Yet, the relationship between driving variables and Earth's surface at such local scales remains unresolved by current physical models. Large Earth observation datasets now enable us to create machine learning models capable of translating coarse weather information into high-resolution Earth surface forecasts. Here, we defi…
Connecting the Dots: A Machine Learning Ready Dataset for Ionospheric Forecasting Models
Department of Information and Computer Science（信息与计算机科学系）；University of Hawai‘i at Mānoa（夏威夷大学毛纳罗亚分校）；Department of Engineering Science（工程科学系）；University of Oxford（牛津大学）；Università degli Studi di Roma Sapienza（罗马大学）；Free Flight Research Lab（自由飞行研究实验室）；University of New Hampshire（新罕布什尔大学）；European Space Agency (ESA)（欧洲航天局）；NASA Jet Propulsion Laboratory（美国宇航局喷气推进实验室）；NASA Headquarters（美国宇航局总部）；Space Weather Technology, Research, and Education Center（空间天气技术、研究与教育中心）；University of Colorado Boulder（科罗拉多大学博尔德分校）；Department of Computer Science（计算机科学系）；University of York & King’s College London（约克大学及伦敦国王学院）
Data-Driven Uncertainty-Aware Forecasting of Sea Ice Conditions in the Gulf of Ob Based on Satellite Radar Imagery
Applied AI Center, Skolkovo Institute of Science and Technology, Moscow, Russia（斯克尔科沃科学与技术研究所应用人工智能中心）；Moscow Institute of Physics and Technology, Dolgoprudny, Russia（莫斯科物理技术学院）；Autonomous Non-Profit Organization Artificial Intelligence Research Institute (AIRI), Moscow, Russia（自主非营利组织人工智能研究 institute（AIRI））
University of Cambridge（剑桥大学）；ENGIE Lab CRIGEN（昂吉集团CRIGEN实验室）
Integration of intermittent renewable energy sources into electric grids in large proportions is challenging. A well-established approach aimed at addressing this difficulty involves the anticipation of the upcoming energy supply variability to adapt the response of the grid. In solar energy, short-term changes in electricity production caused by occluding clouds can be predicted at different time scales from all-sky cameras (up to 30-min ahead) and satellite observations (up to 6h ahead). In this study, we integrate these two complementary points of view on the cloud cover in a single machine…
TAMMs: Change Understanding and Forecasting in Satellite Image Time Series with Temporal-Aware Multimodal Models
School of Computer Science & Technology, Beijing Institute of Technology（计算机科学与技术学院，北京理工大学）；School of Computing, National University of Singapore（计算学院，新加坡国立大学）
Beloit College（贝洛伊特学院）；SSEC, University of Wisconsin, Madison（威斯康星大学麦迪逊分校空间科学与工程中心）；Stanford University（斯坦福大学）
Wildfire forecasting has been one of the most critical tasks that humanities want to thrive. It plays a vital role in protecting human life. Wildfire prediction, on the other hand, is difficult because of its stochastic and chaotic properties. We tackled the problem by interpreting a series of wildfire images as a video and used it to anticipate how the fire would behave in the future. However, creating video prediction models that account for the inherent uncertainty of the future is challenging. The bulk of published attempts is based on stochastic image-autoregressive recurrent networks, wh…
The Pennsylvania State University（宾夕法尼亚州立大学）；Amazon Lab126（亚马逊Lab126）；Accuweather Inc.（Accuweather公司）；University of Almería（阿尔梅里亚大学）
Meteorologists use shapes and movements of clouds in satellite images as indicators of several major types of severe storms. Satellite imaginary data are in increasingly higher resolution, both spatially and temporally, making it impossible for humans to fully leverage the data in their forecast. Automatic satellite imagery analysis methods that can find storm-related cloud patterns as soon as they are detectable are in demand. We propose a machine learning and pattern recognition based approach to detect "comma-shaped" clouds in satellite images, which are specific cloud distribution patterns…
IBM Research（IBM研究院）；IBM T.J. Watson Research Center（IBM托马斯·J·沃森研究中心）
This paper describes a new algorithm for solar energy forecasting from a sequence of Cloud Optical Depth (COD) images. The algorithm is based on the following simple observation: the dynamics of clouds represented by COD images resembles the motion (transport) of a density in a fluid flow. This suggests that, to forecast the motion of COD images, it is sufficient to forecast the flow. The latter, in turn, can be accomplished by fitting a parametric model of the fluid flow to the COD images observed in the past. Namely, the learning phase of the algorithm is composed of the following steps: (i)…
Czech Technical University in Prague（布拉格捷克理工大学）；Meteopress spol s.r.o.（Meteopress有限责任公司）
The short-term prediction of precipitation is critical in many areas of life. Recently, a large body of work was devoted to forecasting radar reflectivity images. The radar images are available only in areas with ground weather radars. Thus, we aim to predict high-resolution precipitation from lower-resolution satellite radiance images. A neural network called WeatherFusionNet is employed to predict severe rain up to eight hours in advance. WeatherFusionNet is a U-Net architecture that fuses three different ways to process the satellite data; predicting future satellite frames, extracting rain…
The Pennsylvania State University（宾夕法尼亚州立大学）；AccuWeather Inc.（AccuWeather公司）
Computers are widely utilized in today's weather forecasting as a powerful tool to leverage an enormous amount of data. Yet, despite the availability of such data, current techniques often fall short of producing reliable detailed storm forecasts. Each year severe thunderstorms cause significant damage and loss of life, some of which could be avoided if better forecasts were available. We propose a computer algorithm that analyzes satellite images from historical archives to locate visual signatures of severe thunderstorms for short-term predictions. While computers are involved in weather for…
Large-scale flood modeling and forecasting with FloodCast
Technical University of Munich（慕尼黑工业大学）；University of Bristol（布里斯托尔大学）；Institute of Mountain Hazards and Environment, Chinese Academy of Sciences（中国科学院成都山地灾害与环境研究所）
Traditional weather forecasting relies on domain expertise and computationally intensive numerical simulation systems. Recently, with the development of a data-driven approach, weather forecasting based on deep learning has been receiving attention. Deep learning-based weather forecasting has made stunning progress, from various backbone studies using CNN, RNN, and Transformer to training strategies using weather observations datasets with auxiliary inputs. All of this progress has contributed to the field of weather forecasting; however, many elements and complex structures of deep learning m…
The supply and demand of energy is influenced by meteorological conditions. The relevance of accurate weather forecasts increases as the demand for renewable energy sources increases. The energy providers and policy makers require weather information to make informed choices and establish optimal plans according to the operational objectives. Due to the recent development of deep learning techniques applied to satellite imagery, weather forecasting that uses remote sensing data has also been the subject of major progress. The present paper investigates multiple steps ahead frame prediction for…
Distributed solar generation forecasting using attention-based deep neural networks for cloud movement prediction
Department of Mechanical Engineering, School of Electrical, Mechanical and Infrastructure Engineering, The University of Melbourne, Melbourne, Australia（机械工程系，电气、机械与基础设施工程学院，墨尔本大学，墨尔本，澳大利亚）；Department of Mechanical Engineering, Sri Lanka Institute of Information Technology, Sri Lanka（机械工程系，斯里兰卡信息科技研究所，斯里兰卡）；School of Computing and Information Systems, Melbourne Centre for Data Science, The University of Melbourne, Melbourne, Australia（计算与信息学院，墨尔本数据科学中心，墨尔本大学，墨尔本，澳大利亚）
Deep Learning Super Resolution for Satellite Cloud Mask Downscaling
BEYOND EO Centre（BEYOND EO中心）；IAASARS, National Observatory of Athens（雅典国家天文台IAASARS研究所）；National and Kapodistrian University of Athens（雅典国立及卡波迪斯特里亚大学）
SI Analytics；NVIDIA（英伟达）；National Institute of Meteorological Sciences (NIMS)（国家气象科学研究所）
Geostationary satellite imagery has applications in climate and weather forecasting, planning natural energy resources, and predicting extreme weather events. For precise and accurate prediction, higher spatial and temporal resolution of geostationary satellite imagery is important. Although recent geostationary satellite resolution has improved, the long-term analysis of climate applications is limited to using multiple satellites from the past to the present due to the different resolutions. To solve this problem, we proposed warp and refine network (WR-Net). WR-Net is divided into an optica…
