The challenge of the cloud removal task can be alleviated with the aid of Synthetic Aperture Radar (SAR) images that can penetrate cloud cover. However, the large domain gap between optical and SAR images as well as the severe speckle noise of SAR images may cause significant interference in SAR-based cloud removal, resulting in performance degeneration. In this paper, we propose a novel global-local fusion based cloud removal (GLF-CR) algorithm to leverage the complementary information embedded in SAR images. Exploiting the power of SAR information to promote cloud removal entails two aspects. The first, global fusion, guides the relationship among all local optical windows to maintain the structure of the recovered region consistent with the remaining cloud-free regions. The second, local fusion, transfers complementary information embedded in the SAR image that corresponds to cloudy areas to generate reliable texture details of the missing regions, and uses dynamic filtering to alleviate the performance degradation caused by speckle noise. Extensive evaluation demonstrates that the proposed algorithm can yield high quality cloud-free images and outperform state-of-the-art cloud removal algorithms with a gain about 1.7dB in terms of PSNR on SEN12MS-CR dataset.
Satellite images are often contaminated by clouds. Cloud removal has received much attention due to the wide range of satellite image applications. As the clouds thicken, the process of removing the clouds becomes more challenging. In such cases, using auxiliary images such as near-infrared or synthetic aperture radar (SAR) for reconstructing is common. In this study, we attempt to solve the problem using two generative adversarial networks (GANs). The first translates SAR images into optical images, and the second removes clouds using the translated images of prior GAN. Also, we propose dilat…
Speckle reduction is a longstanding topic in synthetic aperture radar (SAR) images. Many different schemes have been proposed for the restoration of intensity SAR images. Among the different possible approaches, methods based on convolutional neural networks (CNNs) have recently shown to reach state-of-the-art performance for SAR image restoration. CNN training requires good training data: many pairs of speckle-free / speckle-corrupted images. This is an issue in SAR applications, given the inherent scarcity of speckle-free images. To handle this problem, this paper analyzes different strategi…
SARU: A Shadow-Aware and Removal Unified Framework for Remote Sensing Images with New Benchmarks
MOE Key Lab of ICSP, Anhui Provincial Key Lab of Multimodal Cognitive Computation, IMIS Lab of Anhui Province, School of Computer Science and Technology, Anhui University, Hefei, China（教育部信息与通信系统重点实验室、安徽省多模态认知计算重点实验室、安徽省IMIS实验室、计算机科学与技术学院、安徽大学、合肥，中国）；Graduate School of Frontier Sciences, The University of Tokyo, Chiba, 277-8561, Japan（前沿科学研究生院、东京大学、千叶，日本）
University of Sannio（萨尼奥大学）；Warsaw University of Technology（华沙理工大学）；La Sapienza University（罗马第一大学）；European Space Agency（欧洲空间局）
Cloud removal is a relevant topic in Remote Sensing as it fosters the usability of high-resolution optical images for Earth monitoring and study. Related techniques have been analyzed for years with a progressively clearer view of the appropriate methods to adopt, from multi-spectral to inpainting methods. Recent applications of deep generative models and sequence-to-sequence-based models have proved their capability to advance the field significantly. Nevertheless, there are still some gaps, mostly related to the amount of cloud coverage, the density and thickness of clouds, and the occurred …
Birla Institute of Technology and Science（比尔拉科技与工程学院）
Coherent imaging systems like synthetic aperture radar are susceptible to multiplicative noise that makes applications like automatic target recognition challenging. In this paper, NeighCNN, a deep learning-based speckle reduction algorithm that handles multiplicative noise with relatively simple convolutional neural network architecture, is proposed. We have designed a loss function which is an unique combination of weighted sum of Euclidean, neighbourhood, and perceptual loss for training the deep network. Euclidean and neighbourhood losses take pixel-level information into account, whereas …
Key Lab for Information Science of Electromagnetic Waves (MoE), Fudan University（复旦大学电磁波信息科学教育部重点实验室）
A deep neural networks based method is proposed to convert single polarization grayscale SAR image to fully polarimetric. It consists of two components: a feature extractor network to extract hierarchical multi-scale spatial features of grayscale SAR image, followed by a feature translator network to map spatial feature to polarimetric feature with which the polarimetric covariance matrix of each pixel can be reconstructed. Both qualitative and quantitative experiments with real fully polarimetric data are conducted to show the efficacy of the proposed method. The reconstructed full-pol SAR im…
Enhancing Scene Classification in Cloudy Image Scenarios: A Collaborative Transfer Method with Information Regulation Mechanism using Optical Cloud-Covered and SAR Remote Sensing Images
School of Geosciences and Info-Physics, Central South University（中南大学地球科学与信息物理学院）；Faculty of Geo-Information Science and Earth Observation (ITC), University of Twente（特文特大学地球信息科学与地球观测学院）
In this paper, we present the optical image simulation from a synthetic aperture radar (SAR) data using deep learning based methods. Two models, i.e., optical image simulation directly from the SAR data and from multi-temporal SARoptical data, are proposed to testify the possibilities. The deep learning based methods that we chose to achieve the models are a convolutional neural network (CNN) with a residual architecture and a conditional generative adversarial network (cGAN). We validate our models using the Sentinel-1 and -2 datasets. The experiments demonstrate that the model with multi-tem…
Task-Guided Prompting for Unified Remote Sensing Image Restoration
School of Electronic and Information Engineering, Ningbo University of Technology（宁波工程学院电子与信息工程学院）；University of Exeter（埃克塞特大学）；Engineering Research Center of Intelligent Finance, Ministry of Education（教育部智能金融工程研究中心）；School of Computing and Artificial Intelligence, Southwestern University of Finance and Economics（西南财经大学计算机与人工智能学院）
SAOCOM Mission, National Commission of Space Activities (CONAE)（阿根廷国家空间活动委员会（CONAE）SAOCOM任务）
Synthetic aperture radar (SAR) images are subject to prominent speckle noise, which is generally considered a purely multiplicative noise process. In theory, this multiplicative noise is that the ratio of the standard deviation to the signal value, the "coefficient of variation," is theoretically constant at every point in a SAR image. Most of the filters for speckle reduction are based on this property. Such property is irrelevant for the new filter structure, which is based on directional smoothing (DS) theory, the enhanced directional smoothing (EDS) that removes speckle noise from SAR imag…
SOMA-1M: A Large-Scale SAR-Optical Multi-resolution Alignment Dataset for Multi-Task Remote Sensing
School of Remote Sensing Information Engineering, Wuhan University（遥感信息工程学院，武汉大学）；Hubei LuoJia Laboratory（湖北珞珈实验室）；Technology Innovation Center for Collaborative Applications of Natural Resources Data in GBA, Ministry of Natural Resources（粤港澳大湾区自然资源数据协同应用技术创新中心，自然资源部）
Asynchronous Remote Sensing Time-Series Fusion for Cloud Removal and Anytime Reconstruction
School of Computing and Augmented Intelligence, Arizona State University（计算与增强智能学院，亚利桑那州立大学）；School of Geographical Sciences and Urban Planning, Arizona State University（地理科学与城市规划学院，亚利桑那州立大学）；Woodwell Climate Research Center（伍德沃德气候研究中心）
Nagoya University（名古屋大学）；Chubu University（中部大学）；Tokyo Institute of Technology（东京工业大学）；Advanced Industrial Science and Technology（产业技术综合研究所）
In this paper, we propose a method for cloud removal from visible light RGB satellite images by extending the conditional Generative Adversarial Networks (cGANs) from RGB images to multispectral images. Satellite images have been widely utilized for various purposes, such as natural environment monitoring (pollution, forest or rivers), transportation improvement and prompt emergency response to disasters. However, the obscurity caused by clouds makes it unstable to monitor the situation on the ground with the visible light camera. Images captured by a longer wavelength are introduced to reduce…
