针对 SAR 洪水图像难以人工解读的问题，本文提出基于扩散模型的 DSE 框架，将 SAR 图像转换为 EO 图像，并在两个洪水数据集上验证其视觉与分割性能提升。
摘要：
受快速气候变化驱动，洪水事件的频率和强度正在不断增加。电光（Electro-Optical，EO）卫星图像通常被用于快速响应。然而，其在洪水情境中的效用受到云层覆盖以及夜间限制等问题的阻碍，使准确评估灾害损失变得具有挑战性。已有若干利用合成孔径雷达（Synthetic Aperture Radar，SAR）数据的替代洪水检测技术被提出。尽管在上述情况下 SAR 相比 EO 具有优势，但 SAR 也呈现出一个明显缺点：人类分析人员往往难以解读这些数据。为解决这一问题，本文提出了一个新颖框架，即基于扩散的 SAR 到 EO 图像转换（Diffusion-Based SAR to EO Image Translation，DSE）。DSE 框架将 SAR 图像转换为 EO 图像，从而增强洪水洞察对人类的可解释性。在 Sen1Floods11 和 SEN12-FLOOD 数据集上的实验结果证实，DSE 框架不仅提供了增强的视觉信息，还提升了所有受测洪水分割基线的性能。
英文摘要：
Driven by rapid climate change, the frequency and intensity of flood events are increasing. Electro-Optical (EO) satellite imagery is commonly utilized for rapid response. However, its utilities in flood situations are hampered by issues such as cloud cover and limitations during nighttime, making accurate assessment of damage challenging. Several alternative flood detection techniques utilizing Synthetic Aperture Radar (SAR) data have been proposed. Despite the advantages of SAR over EO in the aforementioned situations, SAR presents a distinct drawback: human analysts often struggle with data interpretation. To tackle this issue, this paper introduces a novel framework, Diffusion-Based SAR to EO Image Translation (DSE). The DSE framework converts SAR images into EO images, thereby enhancing the interpretability of flood insights for humans. Experimental results on the Sen1Floods11 and SEN12-FLOOD datasets confirm that the DSE framework not only delivers enhanced visual information but also improves performance across all tested flood segmentation baselines.
Urban Flood Mapping Using Satellite Synthetic Aperture Radar Data: A Review of Characteristics, Approaches and Datasets
Technical University of Munich（慕尼黑工业大学）；Vienna University of Technology（维也纳工业大学）；ETH Zurich（苏黎世联邦理工学院）；Wuhan University（武汉大学）；DiXplorer AG（4DiXplorer公司）；Luxembourg Institute of Science and Technology (LIST)（卢森堡科学技术研究院）
Due to climate and land-use change, natural disasters such as flooding have been increasing in recent years. Timely and reliable flood detection and mapping can help emergency response and disaster management. In this work, we propose a flood detection network using bi-temporal SAR acquisitions. The proposed segmentation network has an encoder-decoder architecture with two Siamese encoders for pre and post-flood images. The network's feature maps are fused and enhanced using attention blocks to achieve more accurate detection of the flooded areas. Our proposed network is evaluated on publicly …
Flood-DamageSense: Multimodal Mamba with Multitask Learning for Building Flood Damage Assessment using SAR Remote Sensing Imagery
Urban Resilience.AI Lab（城市韧性人工智能实验室）；Zachry Department of Civil and Environmental Engineering（扎克里土木与环境工程系）；Texas A&M University（德克萨斯A&M大学）；College Station, TX（学院站, 德克萨斯）
Human civilization has an increasingly powerful influence on the earth system. Affected by climate change and land-use change, natural disasters such as flooding have been increasing in recent years. Earth observations are an invaluable source for assessing and mitigating negative impacts. Detecting changes from Earth observation data is one way to monitor the possible impact. Effective and reliable Change Detection (CD) methods can help in identifying the risk of disaster events at an early stage. In this work, we propose a novel unsupervised CD method on time series Synthetic Aperture Radar~…
Evaluating the effects of preprocessing, method selection, and hyperparameter tuning on SAR-based flood mapping and water depth estimation
EDF R&D（EDF研究与发展）；Laboratoire National d’Hydraulique et Environnement (LNHE)（国家水利与环境实验室）；ENPC（巴黎综合理工学院）；Institut Polytechnique de Paris（巴黎理工学院）；Inria（法国国家信息与自动化技术研究所）
DAF MIT AI Accelerator（美国空军麻省理工学院人工智能加速器）；MIT Lincoln Laboratory（麻省理工学院林肯实验室）；MIT（麻省理工学院）
This paper explores the use of multi-conditional adversarial networks for SAR-to-EO image translation. Previous methods condition adversarial networks only on the input SAR. We show that incorporating multiple complementary modalities such as Google maps and IR can further improve SAR-to-EO image translation especially on preserving sharp edges of manmade objects. We demonstrate effectiveness of our approach on a diverse set of datasets including SEN12MS, DFC2020, and SpaceNet6. Our experimental results suggest that additional information provided by complementary modalities improves the perfo…
Carted；NVIDIA（英伟达）；Frontier Development Lab（前沿发展实验室）；SpaceML
Floods wreak havoc throughout the world, causing billions of dollars in damages, and uprooting communities, ecosystems and economies. The NASA Impact Flood Detection competition tasked participants with predicting flooded pixels after training with synthetic aperture radar (SAR) images in a supervised setting. We propose a semi-supervised learning pseudo-labeling scheme that derives confidence estimates from U-Net ensembles, progressively improving accuracy. Concretely, we use a cyclical approach involving multiple stages (1) training an ensemble model of multiple U-Net architectures with the …
Mapping Global Floods with 10 Years of Satellite Radar Data
Microsoft AI for Good Research Lab（微软人工智能公益研究院）；Cooperative Institute for Meteorological Satellite Studies, University of Wisconsin-Madison（威斯康星大学麦迪逊分校合作气象卫星研究所）
The increasing intensity and frequency of floods is one of the many consequences of our changing climate. In this work, we explore ML techniques that improve the flood detection module of an operational early flood warning system. Our method exploits an unlabelled dataset of paired multi-spectral and Synthetic Aperture Radar (SAR) imagery to reduce the labeling requirements of a purely supervised learning method. Prior works have used unlabelled data by creating weak labels out of them. However, from our experiments we noticed that such a model still ends up learning the label mistakes in thos…
Improved flood mapping for efficient policy design by fusion of Sentinel-1, Sentinel-2, and Landsat-9 imagery to identify population and infrastructure exposed to floods
Lahore University of Management Sciences (LUMS)（拉合尔管理科学大学（LUMS））
This technical report summarizes the analysis and approach on the image-to-image translation task in the Multimodal Learning for Earth and Environment Challenge (MultiEarth 2022). In terms of strategy optimization, cloud classification is utilized to filter optical images with dense cloud coverage to aid the supervised learning alike approach. The commonly used pix2pix framework with a few optimizations is applied to build the model. A weighted combination of mean squared error and mean absolute error is incorporated in the loss function. As for evaluation, peak to signal ratio and structural …
Unpaired Object-Level SAR-to-Optical Image Translation for Aircraft with Keypoints-Guided Diffusion Models
School of Information Science and Technology, Fudan University（复旦大学信息科学与技术学院）；Key Laboratory for Information Science of Electromagnetic Waves (Ministry of Education)（电磁波信息科学教育部重点实验室）
Satellite images are often contaminated by clouds. Cloud removal has received much attention due to the wide range of satellite image applications. As the clouds thicken, the process of removing the clouds becomes more challenging. In such cases, using auxiliary images such as near-infrared or synthetic aperture radar (SAR) for reconstructing is common. In this study, we attempt to solve the problem using two generative adversarial networks (GANs). The first translates SAR images into optical images, and the second removes clouds using the translated images of prior GAN. Also, we propose dilat…
Seg-CycleGAN : SAR-to-optical image translation guided by a downstream task
Gina Cody School of Engineering and Computer Science, Concordia University（康考迪亚大学吉娜·科迪工程与计算机科学学院）；School of Automation, Northwestern Polytechnical University（西北工业大学自动化学院）；School of Cybersecurity, Northwestern Polytechnical University（西北工业大学网络空间安全学院）
Despite the advantages of all-weather and all-day high-resolution imaging, synthetic aperture radar (SAR) images are much less viewed and used by general people because human vision is not adapted to microwave scattering phenomenon. However, expert interpreters can be trained by comparing side-by-side SAR and optical images to learn the mapping rules from SAR to optical. This paper attempts to develop machine intelligence that are trainable with large-volume co-registered SAR and optical images to translate SAR image to optical version for assisted SAR image interpretation. Reciprocal SAR-Opti…
German Aerospace Center (DLR)（德国航空航天中心）；School of Information and Electronics, Beijing Institute of Technology（北京理工大学信息与电子学院）
Contrary to optical images, Synthetic Aperture Radar (SAR) images are in different electromagnetic spectrum where the human visual system is not accustomed to. Thus, with more and more SAR applications, the demand for enhanced high-quality SAR images has increased considerably. However, high-quality SAR images entail high costs due to the limitations of current SAR devices and their image processing resources. To improve the quality of SAR images and to reduce the costs of their generation, we propose a Dialectical Generative Adversarial Network (Dialectical GAN) to generate high-quality SAR i…
Toshiba Infrastructure Systems & Solutions Corporation（东芝基础设施系统与解决方案公司）
This report deals with translation invariance of convolutional neural networks (CNNs) for automatic target recognition (ATR) from synthetic aperture radar (SAR) imagery. In particular, the translation invariance of CNNs for SAR ATR represents the robustness against misalignment of target chips extracted from SAR images. To understand the translation invariance of the CNNs, we trained CNNs which classify the target chips from the MSTAR into the ten classes under the condition of with and without data augmentation, and then visualized the translation invariance of the CNNs. According to our resu…
Leveraging AI multimodal geospatial foundation models for improved near-real-time flood mapping at a global scale
North Carolina State University（北卡罗来纳州立大学）；LabCorp（实验室集团）；Humboldt-Universität zu Berlin（柏林洪堡大学）；Earth and Life Institute, UCLouvain（鲁汶天主教大学地球与生命研究所）；University of Helsinki（赫尔辛基大学）；Clark University（克拉克大学）；Thünen Institute of Farm Economics（蒂嫩农场经济研究所）
ContraFM-S2O: Flow Matching-Based One-step SAR-to-Optical Image Translation Model with Contrastive Learning
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；The University of Manchester（曼彻斯特大学）；Capital University of Physical Education And Sports（首都体育学院）；Shanghai Jiao Tong University（上海交通大学）
Bridging Modalities and Tasks: A Unified Hierarchical ViT for SAR-to-Optical Translation and Semantic Segmentation
School of Automation, Northwestern Polytechnical University（西北工业大学自动化学院）；School of Cybersecurity, Northwestern Polytechnical University（西北工业大学网络空间安全学院）
Despite the advantages of all-weather and all-day high-resolution imaging, SAR remote sensing images are much less viewed and used by general people because human vision is not adapted to microwave scattering phenomenon. However, expert interpreters can be trained by compare side-by-side SAR and optical images to learn the translation rules from SAR to optical. This paper attempts to develop machine intelligence that are trainable with large-volume co-registered SAR and optical images to translate SAR image to optical version for assisted SAR interpretation. A novel reciprocal GAN scheme is pr…
Swiss Federal Institute of Technology Zurich (ETHZ)（苏黎世联邦理工学院）；Swiss Federal Institute of Aquatic Science and Technology (Eawag)（瑞士联邦水生科学技术研究所）；University of Coimbra（科英布拉大学）
Computational complexity has been the bottleneck of applying physically-based simulations on large urban areas with high spatial resolution for efficient and systematic flooding analyses and risk assessments. To address this issue of long computational time, this paper proposes that the prediction of maximum water depth rasters can be considered as an image-to-image translation problem where the results are generated from input elevation rasters using the information learned from data rather than by conducting simulations, which can significantly accelerate the prediction process. The proposed…
Adversarial Robustness of Deep Learning Models for Inland Water Body Segmentation from SAR Images
Graphics-Visualization-Computing Lab, International Institute of Information Technology Bangalore, Karnataka 560100, India（国际信息学院信息技术研究所，班加罗尔，卡纳塔克邦，印度）；Department of Electronics and Communication Engineering, Manipal Institute of Technology, Manipal Academy of Higher Education, Manipal, Karnataka 576104, India（电子与通信工程系，曼尼普尔理工学院，曼尼普尔高等教育学院，曼尼普尔，卡纳塔克邦，印度）
Multitemporal SAR images change detection and visualization using RABASAR and simplified GLR
Télécom ParisTech（巴黎高等电信学院）；Université Paris-Saclay（巴黎萨克雷大学）；CNRS（法国国家科学研究中心）；Bordeaux INP（波尔多国立理工学院）；University of Bordeaux（波尔多大学）；UJM-Saint-Etienne（圣埃蒂安大学）；Institut d Optique Graduate School（光学研究所研究生院）；University of Lyon（里昂大学）
本文提出基于 RABASAR 去噪与简化广义似然比 $S_{GLR}$ 的多时相 SAR 变化检测框架，并结合变化幅度、谱聚类分类及 REACTIV 可视化，在多类地表变化中表现良好。
Image-to-image (I2I) translation is an established way of translating data from one domain to another but the usability of the translated images in the target domain when working with such dissimilar domains as the SAR/optical satellite imagery ones and how much of the origin domain is translated to the target domain is still not clear enough. This article address this by performing translations of labelled datasets from the optical domain to the SAR domain with different I2I algorithms from the state-of-the-art, learning from transferred features in the destination domain and evaluating later…
University of Kaiserslautern-Landau（凯泽斯劳滕-兰道大学）；German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心）；University of Yamanashi（山梨大学）
Climate change has increased the severity and frequency of weather disasters all around the world. Flood inundation mapping based on earth observation data can help in this context, by providing cheap and accurate maps depicting the area affected by a flood event to emergency-relief units in near-real-time. Building upon the recent development of the Sen1Floods11 dataset, which provides a limited amount of hand-labeled high-quality training data, this paper evaluates the potential of five traditional machine learning approaches such as gradient boosted decision trees, support vector machines o…
Generating Physically-Consistent Satellite Imagery for Climate Visualizations
Massachusetts Institute of Technology（麻省理工学院）；Max Planck Institute for Biogeochemistry（马克斯·普朗克生物地球化学研究所）；University of British Columbia（不列颠哥伦比亚大学）；Peaceful Society, Science and Innovation Foundation（和平协会科学与创新基金会）；University of Granada（格拉纳达大学）；Scripps Institute of Oceanography（斯克里普斯海洋研究所）；US Forest Service（美国林务局）；University of Oxford（牛津大学）；Riot Games（拳头游戏）；Pasteur Labs（巴斯德实验室）；Surgical Data Science Collective（外科数据科学集体）
University of Sannio（萨尼奥大学）；Warsaw University of Technology（华沙理工大学）；La Sapienza University（罗马第一大学）；European Space Agency（欧洲空间局）
Cloud removal is a relevant topic in Remote Sensing as it fosters the usability of high-resolution optical images for Earth monitoring and study. Related techniques have been analyzed for years with a progressively clearer view of the appropriate methods to adopt, from multi-spectral to inpainting methods. Recent applications of deep generative models and sequence-to-sequence-based models have proved their capability to advance the field significantly. Nevertheless, there are still some gaps, mostly related to the amount of cloud coverage, the density and thickness of clouds, and the occurred …
