Accurate precipitation estimation is critical for flood forecasting, water resource management, and disaster preparedness. Satellite products provide global hourly coverage but contain systematic biases; ground-based gauges are accurate at point locations but too sparse for direct gridded correction. Existing methods fuse these sources by interpolating gauge observations onto the satellite grid, but treat each time step independently and therefore discard temporal structure in precipitation fields. We propose Neural Stochastic Process (NSP), a model that pairs a Neural Process encoder conditioning on arbitrary sets of gauge observations with a latent Neural SDE on a 2D spatial representation. NSP is trained under a single variational objective with simulation-free cost. We also introduce QPEBench, a benchmark of 43{,}756 hourly samples over the Contiguous United States (2021--2025) with four aligned data sources and six evaluation metrics. On QPEBench, NSP outperforms 13 baselines across all six metrics and surpasses JAXA's operational gauge-calibrated product. An additional experiment on Kyushu, Japan confirms generalization to a different region with independent data sources.
Rainy: Unlocking Satellite Calibration for Deep Learning in Precipitation
University of YYY（YYY大学）；Institute of WWW（WWW研究院）；Kunming University of Science and Technology（昆明理工大学）；Universiti Malaya（马来亚大学）；Guangdong Ocean University（广东海洋大学）
University of Oxford（牛津大学）；Inria（法国国家信息与自动化研究所）；University College London（伦敦大学学院）；University of Warsaw（华沙大学）
Extreme precipitation events, such as violent rainfall and hail storms, routinely ravage economies and livelihoods around the developing world. Climate change further aggravates this issue. Data-driven deep learning approaches could widen the access to accurate multi-day forecasts, to mitigate against such events. However, there is currently no benchmark dataset dedicated to the study of global precipitation forecasts. In this paper, we introduce \textbf{RainBench}, a new multi-modal benchmark dataset for data-driven precipitation forecasting. It includes simulated satellite data, a selection …
Czech Technical University in Prague（布拉格捷克理工大学）；Meteopress spol s.r.o.（Meteopress有限责任公司）
The short-term prediction of precipitation is critical in many areas of life. Recently, a large body of work was devoted to forecasting radar reflectivity images. The radar images are available only in areas with ground weather radars. Thus, we aim to predict high-resolution precipitation from lower-resolution satellite radiance images. A neural network called WeatherFusionNet is employed to predict severe rain up to eight hours in advance. WeatherFusionNet is a U-Net architecture that fuses three different ways to process the satellite data; predicting future satellite frames, extracting rain…
Politecnico di Torino（都灵理工大学）；Politecnico Interdepartmental Centre for Service Robotics（都灵理工大学跨部门服务机器人中心）；Università degli Studi di Torino（都灵大学）；Institute of Electronics, Computer and Telecommunication Engineering of the National Research Council of Italy（意大利国家研究委员会电子、计算机与电信工程研究所）
Precision agriculture is considered to be a fundamental approach in pursuing a low-input, high-efficiency, and sustainable kind of agriculture when performing site-specific management practices. To achieve this objective, a reliable and updated description of the local status of crops is required. Remote sensing, and in particular satellite-based imagery, proved to be a valuable tool in crop mapping, monitoring, and diseases assessment. However, freely available satellite imagery with low or moderate resolutions showed some limits in specific agricultural applications, e.g., where crops are gr…
Jackson School of Geosciences（杰克逊地球科学学院）；University of Texas at Austin（德克萨斯大学奥斯汀分校）；CIRES, University of Colorado Boulder（科罗拉多大学博尔德分校环境科学研究合作研究所）；NOAA/Physical Sciences Laboratory（美国国家海洋和大气管理局物理科学实验室）；Purdue University（普渡大学）；Cockrell School of Engineering（科克雷尔工程学院）
Urban downscaling is a link to transfer the knowledge from coarser climate information to city scale assessments. These high-resolution assessments need multiyear climatology of past data and future projections, which are complex and computationally expensive to generate using traditional numerical weather prediction models. The city of Austin, Texas, USA has seen tremendous growth in the past decade. Systematic planning for the future requires the availability of fine resolution city-scale datasets. In this study, we demonstrate a novel approach generating a general purpose operator using dee…
Department of Computer and Electrical Engineering, Rutgers University（计算机与电气工程系，罗格斯大学）；Cloud to Street（云街（Cloud to Street））
Accurate flood detection in near real time via high resolution, high latency satellite imagery is essential to prevent loss of lives by providing quick and actionable information. Instruments and sensors useful for flood detection are only available in low resolution, low latency satellites with region re-visit periods of up to 16 days, making flood alerting systems that use such satellites unreliable. This work presents H2O-Network, a self supervised deep learning method to segment floods from satellites and aerial imagery by bridging domain gap between low and high latency satellite and coar…
From Swath to Full-Disc: Advancing Precipitation Retrieval with Multimodal Knowledge Expansion
College of Computer Science, Zhejiang University of Technology（浙江工业大学计算机科学学院）；National Meteorological Information Center（国家气象信息中心）；College of Computer Science, Zhejiang University of Technology& Zhejiang Key Laboratory of Visual Information Intelligent Processing（浙江工业大学计算机科学学院&浙江视觉信息智能处理重点实验室）
Precipitation nowcasting is a short-range forecast of rain/snow (up to 2 hours), often displayed on top of the geographical map by the weather service. Modern precipitation nowcasting algorithms rely on the extrapolation of observations by ground-based radars via optical flow techniques or neural network models. Dependent on these radars, typical nowcasting is limited to the regions around their locations. We have developed a method for precipitation nowcasting based on geostationary satellite imagery and incorporated the resulting data into the Yandex.Weather precipitation map (including an a…
How Effective Are Time-Series Models for Precipitation Nowcasting? A Comprehensive Benchmark for GNSS-based Precipitation Nowcasting
Sanya Science and Education Innovation Park, Wuhan University of Technology（武汉理工大学三亚科技创新园）；Wuhan University of Technology（武汉理工大学）；School of Computer Science and Artificial Intelligence, Wuhan University of Technology（武汉理工大学计算机科学与人工智能学院）；Interdisciplinary Artificial Intelligence Research Institute, Wuhan College（武汉学院交叉人工智能研究院）；Shanghai Artificial Intelligence Laboratory, Shanghai（上海人工智能实验室）；School of Earth and Space Science and Technology, Wuhan University（武汉大学地球空间科学与技术学院）；School of Water Resources and Hydropower Engineering, Wuhan University（武汉大学水利水电学院）；GNSS Research Center, Wuhan University（武汉大学GNSS研究中心）
Precipitation is a key part of hydrological circulation and is a sensitive indicator of climate change. The Integrated Multi-satellitE Retrievals for the Global Precipitation Measurement (GPM) mission (IMERG) datasets are widely used for global and regional precipitation investigations. However, their local application is limited by the relatively coarse spatial resolution. Therefore, in this paper, an attention mechanism based convolutional network (AMCN) is proposed to downscale GPM IMERG monthly precipitation data. The proposed method is an end-to-end network, which consists of a global cro…
Precipitation nowcasting using neural networks and ground-based radars has become one of the key components of modern weather prediction services, but it is limited to the regions covered by ground-based radars. Truly global precipitation nowcasting requires fusion of radar and satellite observations. We propose the data fusion pipeline based on computer vision techniques, including novel inpainting algorithm with soft masking.
Accurate and timely estimation of precipitation is critical for issuing hazard warnings (e.g., for flash floods or landslides). Current remotely sensed precipitation products have a few hours of latency, associated with the acquisition and processing of satellite data. By applying a robust nowcasting system to these products, it is (in principle) possible to reduce this latency and improve their applicability, value, and impact. However, the development of such a system is complicated by the chaotic nature of the atmosphere, and the consequent rapid changes that can occur in the structures of …
ETH Zurich（苏黎世联邦理工学院）；University of Stuttgart（斯图加特大学）
Modern high-resolution satellite sensors collect optical imagery with ground sampling distances (GSDs) of 30-50cm, which has sparked a renewed interest in photogrammetric 3D surface reconstruction from satellite data. State-of-the-art reconstruction methods typically generate 2.5D elevation data. Here, we present an approach to recover full 3D surface meshes from multi-view satellite imagery. The proposed method takes as input a coarse initial mesh and refines it by iteratively updating all vertex positions to maximize the photo-consistency between images. Photo-consistency is measured in imag…
The Pennsylvania State University（宾夕法尼亚州立大学）；University of California, Irvine（加州大学欧文分校）
Being able to effectively identify clouds and monitor their evolution is one important step toward more accurate quantitative precipitation estimation and forecast. In this study, a new gradient-based cloud-image segmentation technique is developed using tools from image processing techniques. This method integrates morphological image gradient magnitudes to separable cloud systems and patches boundaries. A varying scale-kernel is implemented to reduce the sensitivity of image segmentation to noise and capture objects with various finenesses of the edges in remote-sensing images. The proposed …
We propose an approach to estimate the 6DOF pose of a satellite, relative to a canonical pose, from a single image. Such a problem is crucial in many space proximity operations, such as docking, debris removal, and inter-spacecraft communications. Our approach combines machine learning and geometric optimisation, by predicting the coordinates of a set of landmarks in the input image, associating the landmarks to their corresponding 3D points on an a priori reconstructed 3D model, then solving for the object pose using non-linear optimisation. Our approach is not only novel for this specific po…
Robust RPC Bundle Adjustment for Multi-Date Satellite Imagery with Season-Invariant Correspondences
Eurecat（欧洲电信技术研究中心）；Universidad de la República（乌拉圭共和国大学）；AMIAD；Institut Universitaire de France（法国大学研究院）；Université Paris-Saclay（巴黎萨克雷大学）；CNRS（法国国家科学研究中心）；ENS Paris-Saclay（巴黎萨克雷高等师范学校）；Centre Borelli（博雷利中心）
VMU-Diff: A Coarse-to-fine Multi-source Data Fusion Framework for Precipitation Nowcasting
Department of Automation, Southeast University（东南大学自动化部门）；State Key Laboratory of Virtual Reality Technology and Systems, Beihang University（北京航空航天大学虚拟现实技术与系统国家重点实验室）；Key Laboratory of Transportation Meteorology, China Meteorological Administration（中国气象局交通运输气象重点实验室）
In this paper we present a method for building boundary refinement and regularization in satellite images using a fully convolutional neural network trained with a combination of adversarial and regularized losses. Compared to a pure Mask R-CNN model, the overall algorithm can achieve equivalent performance in terms of accuracy and completeness. However, unlike Mask R-CNN that produces irregular footprints, our framework generates regularized and visually pleasing building boundaries which are beneficial in many applications.
