Deep generative models are increasingly used to gain insights in the geospatial data domain, e.g., for climate data. However, most existing approaches work with temporal snapshots or assume 1D time-series; few are able to capture spatio-temporal processes simultaneously. Beyond this, Earth-systems data often exhibit highly irregular and complex patterns, for example caused by extreme weather events. Because of climate change, these phenomena are only increasing in frequency. Here, we proposed a novel GAN-based approach for generating spatio-temporal weather patterns conditioned on detected extreme events. Our approach augments GAN generator and discriminator with an encoded extreme weather event segmentation mask. These segmentation masks can be created from raw input using existing event detection frameworks. As such, our approach is highly modular and can be combined with custom GAN architectures. We highlight the applicability of our proposed approach in experiments with real-world surface radiation and zonal wind data.
4LC: 4D Latent Cache for Efficient Conditioning in Dynamic World Modeling
Shanghai Jiaotong University（上海交通大学）；Zhongguancun Academy（中关村学院）；Institute of Automation, Chinese Academy of Science（中国科学院自动化研究所）；Harbin Institute of Technology（哈尔滨工业大学）
DAWP: A framework for global observation forecasting via Data Assimilation and Weather Prediction in satellite observation space
Shanghai Jiao Tong University（上海交通大学）；Fudan University（复旦大学）；The Chinese University of Hong Kong Shanghai AI Laboratory（香港中文大学上海人工智能实验室）；Shanghai AI Laboratory（上海人工智能实验室）
Automatic analysis of spatio-temporal microscopy images is inevitable for state-of-the-art research in the life sciences. Recent developments in deep learning provide powerful tools for automatic analyses of such image data, but heavily depend on the amount and quality of provided training data to perform well. To this end, we developed a new method for realistic generation of synthetic 2D+t microscopy image data of fluorescently labeled cellular nuclei. The method combines spatiotemporal statistical shape models of different cell cycle stages with a conditional GAN to generate time series of …
Meteorological satellite imagery is critical for meteorologists. The data have played an important role in monitoring and analyzing weather and climate changes. However, satellite imagery is a kind of observation data and exists a significant time delay when transmitting the data back to Earth. It is important to make accurate predictions for meteorological satellite images, especially the nowcasting prediction up to 2 hours ahead. In recent years, there has been growing interest in the research of nowcasting prediction applications of weather radar images based on deep learning. Compared to t…
AsyncCouple-Flow: Asynchronous Cross-Modal Coupling and Flow Matching for Spatio-Temporal Forecasting
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；Emory University（埃默里大学）；University of California, Berkeley（加州大学伯克利分校）；Yale University（耶鲁大学）；Stevens Institute of Technology（史蒂文斯理工学院）；City University of New York（纽约城市大学）
Identifiable Autoregressive Variational Autoencoders for Nonlinear and Nonstationary Spatio-Temporal Blind Source Separation
Department of Mathematics and Statistics, University of Jyvaskyla, Finland（数学与统计学系，于韦斯屈埃理工大学，芬兰）；Department of Mathematics and Statistics, University of Helsinki, Finland（数学与统计学系，赫尔辛基大学，芬兰）
Rice University（莱斯大学）；University of Pennsylvania（宾夕法尼亚大学）
Convolutional neural networks (CNNs) can potentially provide powerful tools for classifying and identifying patterns in climate and environmental data. However, because of the inherent complexities of such data, which are often spatio-temporal, chaotic, and non-stationary, the CNN algorithms must be designed/evaluated for each specific dataset and application. Yet to start, CNN, a supervised technique, requires a large labeled dataset. Labeling demands (human) expert time, which combined with the limited number of relevant examples in this area, can discourage using CNNs for new problems. To a…
Earth Observatory is a growing research area that can capitalize on the powers of AI for short time forecasting, a Now-casting scenario. In this work, we tackle the challenge of weather forecasting using a video transformer network. Vision transformer architectures have been explored in various applications, with major constraints being the computational complexity of Attention and the data hungry training. To address these issues, we propose the use of Video Swin-Transformer, coupled with a dedicated augmentation scheme. Moreover, we employ gradual spatial reduction on the encoder side and cr…
This paper presents a solution to the Weather4cast 2022 Challenge Stage 2. The goal of the challenge is to forecast future high-resolution rainfall events obtained from ground radar using low-resolution multiband satellite images. We suggest a solution that performs data preprocessing appropriate to the challenge and then predicts rainfall movies using a novel RainUNet. RainUNet is a hierarchical U-shaped network with temporal-wise separable block (TS block) using a decoupled large kernel 3D convolution to improve the prediction performance. Various evaluation metrics show that our solution is…
Global spatio-temporal downscaling of ERA5 precipitation through generative AI
Institute of Meteorology and Climate Research - Atmospheric Environmental Research (IMK-IFU)（气象与气候研究所——大气环境研究（IMK-IFU））；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；Institute of Meteorology and Climate Research - Atmospheric Trace Gases and Remote Sensing (IMK-ASF)（气象与气候研究所——大气痕量气体与遥感（IMK-ASF））；University of Augsburg（奥格斯堡大学）
MeteorPred: A Meteorological Multimodal Large Model and Dataset for Severe Weather Event Prediction
MAIS, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所MAIS）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Zhongguancun Academy, Beijing（中关村学院）；China Meteorological Administration（中国气象局）
UMass Amherst（马萨诸塞大学阿默斯特分校）；University of Washington（华盛顿大学）；UNC Asheville（北卡罗来纳大学阿什维尔分校）；Cornell University（康奈尔大学）
The US weather radar archive holds detailed information about biological phenomena in the atmosphere over the last 20 years. Communally roosting birds congregate in large numbers at nighttime roosting locations, and their morning exodus from the roost is often visible as a distinctive pattern in radar images. This paper describes a machine learning system to detect and track roost signatures in weather radar data. A significant challenge is that labels were collected opportunistically from previous research studies and there are systematic differences in labeling style. We contribute a latent …
GMG: A Video Prediction Method Based on Global Focus and Motion Guided
College of Atmospheric Sciences, Lanzhou University（兰州大学大气科学学院）；College of Computer and Mathematics, Central South University of Forestry and Technology（中央林业科技大学计算机与数学学院）；Center for Language and Information Processing, University of Munich (LMU)（慕尼黑大学语言与信息处理中心）；Department of Computer Science, University of Manchester（曼彻斯特大学计算机科学系）
Farm parcel delineation provides cadastral data that is important in developing and managing climate change policies. Specifically, farm parcel delineation informs applications in downstream governmental policies of land allocation, irrigation, fertilization, green-house gases (GHG's), etc. This data can also be useful for the agricultural insurance sector for assessing compensations following damages associated with extreme weather events - a growing trend related to climate change. Using satellite imaging can be a scalable and cost effective manner to perform the task of farm parcel delineat…
MILA, Université de Montréal（蒙特利尔大学 MILA）；Lawrence Berkeley National Lab（劳伦斯伯克利国家实验室）；École Polytechnique de Montréal（蒙特利尔理工学院）；Microsoft Maluuba（微软 Maluuba）
Then detection and identification of extreme weather events in large-scale climate simulations is an important problem for risk management, informing governmental policy decisions and advancing our basic understanding of the climate system. Recent work has shown that fully supervised convolutional neural networks (CNNs) can yield acceptable accuracy for classifying well-known types of extreme weather events when large amounts of labeled data are available. However, many different types of spatially localized climate patterns are of interest including hurricanes, extra-tropical cyclones, weathe…
