Earth Observatory is a growing research area that can capitalize on the powers of AI for short time forecasting, a Now-casting scenario. In this work, we tackle the challenge of weather forecasting using a video transformer network. Vision transformer architectures have been explored in various applications, with major constraints being the computational complexity of Attention and the data hungry training. To address these issues, we propose the use of Video Swin-Transformer, coupled with a dedicated augmentation scheme. Moreover, we employ gradual spatial reduction on the encoder side and cross-attention on the decoder. The proposed approach is tested on the Weather4Cast2021 weather forecasting challenge data, which requires the prediction of 8 hours ahead future frames (4 per hour) from an hourly weather product sequence. The dataset was normalized to 0-1 to facilitate using the evaluation metrics across different datasets. The model results in an MSE score of 0.4750 when provided with training data, and 0.4420 during transfer learning without using training data, respectively.
Traditional weather forecasting relies on domain expertise and computationally intensive numerical simulation systems. Recently, with the development of a data-driven approach, weather forecasting based on deep learning has been receiving attention. Deep learning-based weather forecasting has made stunning progress, from various backbone studies using CNN, RNN, and Transformer to training strategies using weather observations datasets with auxiliary inputs. All of this progress has contributed to the field of weather forecasting; however, many elements and complex structures of deep learning m…
Precipitation nowcasting using neural networks and ground-based radars has become one of the key components of modern weather prediction services, but it is limited to the regions covered by ground-based radars. Truly global precipitation nowcasting requires fusion of radar and satellite observations. We propose the data fusion pipeline based on computer vision techniques, including novel inpainting algorithm with soft masking.
Traffic forecasting is an important element of mobility management, an important key that drives the logistics industry. Over the years, lots of work have been done in Traffic forecasting using time series as well as spatiotemporal dynamic forecasting. In this paper, we explore the use of vision transformer in a UNet setting. We completely remove all convolution-based building blocks in UNet, while using 3D shifted window transformer in both encoder and decoder branches. In addition, we experiment with the use of feature mixing just before patch encoding to control the inter-relationship of th…
Southwestern University of Finance and Economics（西南财经大学）
Bike-sharing systems (BSSs) have become increasingly popular around the globe and have attracted a wide range of research interests. In this paper, the demand forecasting problem in BSSs is studied. Spatial and temporal features are critical for demand forecasting in BSSs, but it is challenging to extract spatiotemporal dynamics. Another challenge is to capture the relations between spatiotemporal dynamics and external factors, such as weather, day-of-week, and time-of-day. To address these challenges, we propose a multiple spatiotemporal fusion network named MSTF-Net. MSTF-Net consists of mul…
Video Adverse-Weather-Component Suppression Network via Weather Messenger and Adversarial Backpropagation
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The Hong Kong University of Science and Technology（香港科技大学）；University of Cambridge（剑桥大学）；Institute of High Performance Computing, Agency for Science, Technology and Research（新加坡科技研究局高性能计算研究所）；Tianjin University（天津大学）；Hong Kong Metropolitan University（香港都会大学）
GMG: A Video Prediction Method Based on Global Focus and Motion Guided
College of Atmospheric Sciences, Lanzhou University（兰州大学大气科学学院）；College of Computer and Mathematics, Central South University of Forestry and Technology（中央林业科技大学计算机与数学学院）；Center for Language and Information Processing, University of Munich (LMU)（慕尼黑大学语言与信息处理中心）；Department of Computer Science, University of Manchester（曼彻斯特大学计算机科学系）
Hong Kong University of Science and Technology（香港科技大学）；Hong Kong Observatory（香港天文台）
The goal of precipitation nowcasting is to predict the future rainfall intensity in a local region over a relatively short period of time. Very few previous studies have examined this crucial and challenging weather forecasting problem from the machine learning perspective. In this paper, we formulate precipitation nowcasting as a spatiotemporal sequence forecasting problem in which both the input and the prediction target are spatiotemporal sequences. By extending the fully connected LSTM (FC-LSTM) to have convolutional structures in both the input-to-state and state-to-state transitions, we …
WaveSFNet: A Wavelet-Based Codec and Spatial--Frequency Dual-Domain Gating Network for Spatiotemporal Prediction
College of Software Engineering, Sichuan University（四川大学软件工程学院）；College of Computer Science, Sichuan University（四川大学计算机科学学院）；Sichuan University National Key Laboratory of Fundamental Science on Synthetic Vision（四川大学合成视觉基础科学国家重点实验室）
Meteorological satellite imagery is critical for meteorologists. The data have played an important role in monitoring and analyzing weather and climate changes. However, satellite imagery is a kind of observation data and exists a significant time delay when transmitting the data back to Earth. It is important to make accurate predictions for meteorological satellite images, especially the nowcasting prediction up to 2 hours ahead. In recent years, there has been growing interest in the research of nowcasting prediction applications of weather radar images based on deep learning. Compared to t…
MeteorPred: A Meteorological Multimodal Large Model and Dataset for Severe Weather Event Prediction
MAIS, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所MAIS）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Zhongguancun Academy, Beijing（中关村学院）；China Meteorological Administration（中国气象局）
MILA, Université de Montréal（蒙特利尔大学 MILA）；Lawrence Berkeley National Lab（劳伦斯伯克利国家实验室）；École Polytechnique de Montréal（蒙特利尔理工学院）；Microsoft Maluuba（微软 Maluuba）
Then detection and identification of extreme weather events in large-scale climate simulations is an important problem for risk management, informing governmental policy decisions and advancing our basic understanding of the climate system. Recent work has shown that fully supervised convolutional neural networks (CNNs) can yield acceptable accuracy for classifying well-known types of extreme weather events when large amounts of labeled data are available. However, many different types of spatially localized climate patterns are of interest including hurricanes, extra-tropical cyclones, weathe…
MCSDNet: Mesoscale Convective System Detection Network via Multi-scale Spatiotemporal Information
School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳）计算机科学与技术学院）；Beijing Jinkai New Energy Environmental Technology Co., Ltd.（北京金凯新能源环境技术有限公司）
Uncertainty-Aware Hourly Air Temperature Mapping at 2 km Resolution via Physics-Guided Deep Learning
Spatial Sciences Institute（空间科学研究所）；Dornsife College of Letters, Arts and Sciences（文理学院）；University of Southern California（南加州大学）；Division of Biostatistics, Department of Population and Public Health Sciences（流行病学与公共卫生科学系生物统计学部）；Keck School of Medicine（凯克医学院）
We adapt image inpainting techniques to impute large, irregular missing regions in urban settings characterized by sparsity, variance in both space and time, and anomalous events. Missing regions in urban data can be caused by sensor or software failures, data quality issues, interference from weather events, incomplete data collection, or varying data use regulations; any missing data can render the entire dataset unusable for downstream applications. To ensure coverage and utility, we adapt computer vision techniques for image inpainting to operate on 3D histograms (2D space + 1D time) commo…
Frequency-Aligned Knowledge Distillation for Lightweight Spatiotemporal Forecasting
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；The University of Hong Kong（香港大学）；The City University of New York（纽约城市大学）；Tsinghua University（清华大学）
Ocean University of China（中国海洋大学）；National Marine Environment Forecasting Center（国家海洋环境预报中心）；University of Portsmouth（朴茨茅斯大学）
Forecasts by the European Centre for Medium-Range Weather Forecasts (ECMWF; EC for short) can provide a basis for the establishment of maritime-disaster warning systems, but they contain some systematic biases.The fifth-generation EC atmospheric reanalysis (ERA5) data have high accuracy, but are delayed by about 5 days. To overcome this issue, a spatiotemporal deep-learning method could be used for nonlinear mapping between EC and ERA5 data, which would improve the quality of EC wind forecast data in real time. In this study, we developed the Multi-Task-Double Encoder Trajectory Gated Recurren…
School of Business Administration, Faculty of Business Administration, Southwestern University of Finance and Economics（西南财经大学工商管理学院）
Traffic flow forecasting is essential and challenging to intelligent city management and public safety. Recent studies have shown the potential of convolution-free Transformer approach to extract the dynamic dependencies among complex influencing factors. However, two issues prevent the approach from being effectively applied in traffic flow forecasting. First, it ignores the spatiotemporal structure of the traffic flow videos. Second, for a long sequence, it is hard to focus on crucial attention due to the quadratic times dot-product computation. To address the two issues, we first factorize …
Time Distributed Deep Learning Models for Purely Exogenous Forecasting: Application to Water Table Depth Prediction using Weather Image Time Series
Computer Science Department - University of Turin（都灵大学计算机科学系）；LISTIC Laboratory - Université Savoie Mont Blanc（萨瓦伊-蒙布莱大学LISTIC实验室）；Interuniversity Department of Regional and Urban Studies and Planning, Politecnico di Torino and University of Turin（都灵理工学院和都灵大学区域与城市研究与规划联合学院）
