Although Generative Adversarial Networks (GANs) are successfully applied to diverse fields, training GANs on synthetic aperture radar (SAR) data is a challenging task mostly due to speckle noise. On the one hands, in a learning perspective of human's perception, it is natural to learn a task by using various information from multiple sources. However, in the previous GAN works on SAR target image generation, the information on target classes has only been used. Due to the backscattering characteristics of SAR image signals, the shapes and structures of SAR target images are strongly dependent on their pose angles. Nevertheless, the pose angle information has not been incorporated into such generative models for SAR target images. In this paper, we firstly propose a novel GAN-based multi-task learning (MTL) method for SAR target image generation, called PeaceGAN that uses both pose angle and target class information, which makes it possible to produce SAR target images of desired target classes at intended pose angles. For this, the PeaceGAN has two additional structures, a pose estimator and an auxiliary classifier, at the side of its discriminator to combine the pose and class information more efficiently. In addition, the PeaceGAN is jointly learned in an end-to-end manner as MTL with both pose angle and target class information, thus enhancing the diversity and quality of generated SAR target images The extensive experiments show that taking an advantage of both pose angle and target class learning by the proposed pose estimator and auxiliary classifier can help the PeaceGAN's generator effectively learn the distributions of SAR target images in the MTL framework, so that it can better generate the SAR target images more flexibly and faithfully at intended pose angles for desired target classes compared to the recent state-of-the-art methods.
$\mathbfΦ$-GAN: Physics-Inspired GAN for Generating SAR Images Under Limited Data
School of Automation, Northwestern Polytechnical University（西北工业大学自动化学院）；College of Electronic and Information Engineering, Nanjing University of Aeronautics and Astronautics（南京航空航天大学电子信息工程学院）
Multi-task SAR Image Processing via GAN-based Unsupervised Manipulation
School of Electronic Engineering, Xidian University（西安电子科技大学电子工程学院）；Kunshan Innovation Institute of Xidian University（西安电子科技大学昆山创新研究院）；University of Montenegro（黑山大学）
Polarimetric synthetic aperture radar (PolSAR) images are widely used in disaster detection and military reconnaissance and so on. However, their interpretation faces some challenges, e.g., deficiency of labeled data, inadequate utilization of data information and so on. In this paper, a complex-valued generative adversarial network (GAN) is proposed for the first time to address these issues. The complex number form of model complies with the physical mechanism of PolSAR data and in favor of utilizing and retaining amplitude and phase information of PolSAR data. GAN architecture and semi-supe…
German Aerospace Center (DLR)（德国航空航天中心）；School of Information and Electronics, Beijing Institute of Technology（北京理工大学信息与电子学院）
Contrary to optical images, Synthetic Aperture Radar (SAR) images are in different electromagnetic spectrum where the human visual system is not accustomed to. Thus, with more and more SAR applications, the demand for enhanced high-quality SAR images has increased considerably. However, high-quality SAR images entail high costs due to the limitations of current SAR devices and their image processing resources. To improve the quality of SAR images and to reduce the costs of their generation, we propose a Dialectical Generative Adversarial Network (Dialectical GAN) to generate high-quality SAR i…
Satellite images are often contaminated by clouds. Cloud removal has received much attention due to the wide range of satellite image applications. As the clouds thicken, the process of removing the clouds becomes more challenging. In such cases, using auxiliary images such as near-infrared or synthetic aperture radar (SAR) for reconstructing is common. In this study, we attempt to solve the problem using two generative adversarial networks (GANs). The first translates SAR images into optical images, and the second removes clouds using the translated images of prior GAN. Also, we propose dilat…
Consistency-Regularized GAN for Few-Shot SAR Target Recognition
College of Electronics and Information Engineering, Wuyi University（威夷大学电子与信息工程学院）；College of Electronic and Information Engineering, South China University of Technology（华南理工大学电子与信息工程学院）；Department of Geography, The University of Hong Kong（香港大学地理系）；Faculty of Innovation Engineering, Macau University of Science and Technology（澳门科技大学创新工程学院）；Faculty of Computer Science and Engineering, South China University of Technology（华南理工大学计算机科学与工程学院）
Xidian University（西安电子科技大学）；University of Montenegro（黑山大学）
Generative Adversarial Networks (GANs) can synthesize abundant photo-realistic synthetic aperture radar (SAR) images. Some recent GANs (e.g., InfoGAN), are even able to edit specific properties of the synthesized images by introducing latent codes. It is crucial for SAR image synthesis since the targets in real SAR images are with different properties due to the imaging mechanism. Despite the success of InfoGAN in manipulating properties, there still lacks a clear explanation of how these latent codes affect synthesized properties, thus editing specific properties usually relies on empirical t…
Rutgers, The State University of New Jersey（罗格斯大学新泽西州立大学）
We propose a novel approach for generating high quality visible-like images from Synthetic Aperture Radar (SAR) images using Deep Convolutional Generative Adversarial Network (GAN) architectures. The proposed approach is based on a cascaded network of convolutional neural nets (CNNs) for despeckling and image colorization. The cascaded structure results in faster convergence during training and produces high quality visible images from the corresponding SAR images. Experimental results on both simulated and real SAR images show that the proposed method can produce visible-like images better co…
German Aerospace Center (DLR)（德国航空航天中心）；ETH Zurich（苏黎世联邦理工学院）；Technische Universitaet Muenchen (TUM)（慕尼黑工业大学）
Very High Spatial Resolution (VHSR) large-scale SAR image databases are still an unresolved issue in the Remote Sensing field. In this work, we propose such a dataset and use it to explore patch-based classification in urban and periurban areas, considering 7 distinct semantic classes. In this context, we investigate the accuracy of large CNN classification models and pre-trained networks for SAR imaging systems. Furthermore, we propose a Generative Adversarial Network (GAN) for SAR image generation and test, whether the synthetic data can actually improve classification accuracy.
Despite the advantages of all-weather and all-day high-resolution imaging, synthetic aperture radar (SAR) images are much less viewed and used by general people because human vision is not adapted to microwave scattering phenomenon. However, expert interpreters can be trained by comparing side-by-side SAR and optical images to learn the mapping rules from SAR to optical. This paper attempts to develop machine intelligence that are trainable with large-volume co-registered SAR and optical images to translate SAR image to optical version for assisted SAR image interpretation. Reciprocal SAR-Opti…
Despite the advantages of all-weather and all-day high-resolution imaging, SAR remote sensing images are much less viewed and used by general people because human vision is not adapted to microwave scattering phenomenon. However, expert interpreters can be trained by compare side-by-side SAR and optical images to learn the translation rules from SAR to optical. This paper attempts to develop machine intelligence that are trainable with large-volume co-registered SAR and optical images to translate SAR image to optical version for assisted SAR interpretation. A novel reciprocal GAN scheme is pr…
Nanjing University of Science and Technology（南京理工大学）；Northwestern Polytechnical University（西北工业大学）
Deep learning based semantic segmentation is one of the popular methods in remote sensing image segmentation. In this paper, a network based on the widely used encoderdecoder architecture is proposed to accomplish the synthetic aperture radar (SAR) images segmentation. With the better representation capability of optical images, we propose to enrich SAR images with generated optical images via the generative adversative network (GAN) trained by numerous SAR and optical images. These optical images can be used as expansions of original SAR images, thus ensuring robust result of segmentation. Th…
Computed tomography (CT) uses X-ray measurements taken from sensors around the body to generate tomographic images of the human body. Conventional reconstruction algorithms can be used if the X-ray data are adequately sampled and of high quality; however, concerns such as reducing dose to the patient, or geometric limitations on data acquisition, may result in low quality or incomplete data. Images reconstructed from these data using conventional methods are of poor quality, due to noise and other artifacts. The aim of this study is to train a single neural network to reconstruct high-quality …
ContraFM-S2O: Flow Matching-Based One-step SAR-to-Optical Image Translation Model with Contrastive Learning
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；The University of Manchester（曼彻斯特大学）；Capital University of Physical Education And Sports（首都体育学院）；Shanghai Jiao Tong University（上海交通大学）
RIKEN Center for Advanced Intelligence Project（理化学研究所先进智能项目中心）；Inria（法国国家信息与自动化研究所）；Munich University of Applied Sciences（慕尼黑应用科学大学）；the University of Tokyo（东京大学）
We synthesize both optical RGB and synthetic aperture radar (SAR) remote sensing images from land cover maps and auxiliary raster data using generative adversarial networks (GANs). In remote sensing, many types of data, such as digital elevation models (DEMs) or precipitation maps, are often not reflected in land cover maps but still influence image content or structure. Including such data in the synthesis process increases the quality of the generated images and exerts more control on their characteristics. Spatially adaptive normalization layers fuse both inputs and are applied to a full-bl…
Boston University（波士顿大学）；Microsoft Research（微软研究院）
Although deep neural networks have achieved great performance on classification tasks, recent studies showed that well trained networks can be fooled by adding subtle noises. This paper introduces a new approach to improve neural network robustness by applying the recovery process on top of the naturally trained classifier. In this approach, images will be intentionally corrupted by some significant operator and then be recovered before passing through the classifiers. SARGAN -- an extension on Generative Adversarial Networks (GAN) is capable of denoising radar signals. This paper will show th…
Fast Fourier Convolutional GAN for 30 m Clear-Sky Land Surface Temperature Gap-Free Reconstruction
Technical University of Munich（慕尼黑工业大学）；Helmholtz Centre for Polar and Marine Research, Alfred Wegener Institute（亥姆霍兹极地与海洋研究中心阿尔弗雷德·韦格纳研究所）；Swiss Federal Institute of Technology Zurich (ETH Zurich)（瑞士联邦理工学院苏黎世分校）
Maharaja Agrasen Institute of Technology（马哈拉贾·阿格拉森技术学院）；Near East University（近东大学）；State University of Ceará（塞阿拉州立大学）；University of Fortaleza（福塔莱萨大学）
Coronavirus (COVID-19) is a viral disease caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). The spread of COVID-19 seems to have a detrimental effect on the global economy and health. A positive chest X-ray of infected patients is a crucial step in the battle against COVID-19. Early results suggest that abnormalities exist in chest X-rays of patients suggestive of COVID-19. This has led to the introduction of a variety of deep learning systems and studies have shown that the accuracy of COVID-19 patient detection through the use of chest X-rays is strongly optimistic. Dee…
Exploiting generative self-supervised learning for the assessment of biological images with lack of annotations: a COVID-19 case-study
Politecnico di Torino（都灵理工大学）；Università degli Studi di Torino（都灵大学）；Fondazione Toscana Life Sciences（托斯卡纳生命科学基金会）；Università di Modena and Reggio Emilia（摩德纳-雷焦·艾米里亚大学）
SARS-CoV-2, also known as COVID-19 or Coronavirus, is a viral contagious disease that is infected by a novel coronavirus, and has been rapidly spreading across the globe. It is very important to test and isolate people to reduce spread, and from here comes the need to do this quickly and efficiently. According to some studies, Chest-CT outperforms RT-PCR lab testing, which is the current standard, when diagnosing COVID-19 patients. Due to this, computer vision researchers have developed various deep learning systems that can predict COVID-19 using a Chest-CT scan correctly to a certain degree.…
Seg-CycleGAN : SAR-to-optical image translation guided by a downstream task
Gina Cody School of Engineering and Computer Science, Concordia University（康考迪亚大学吉娜·科迪工程与计算机科学学院）；School of Automation, Northwestern Polytechnical University（西北工业大学自动化学院）；School of Cybersecurity, Northwestern Polytechnical University（西北工业大学网络空间安全学院）
Redwood Center for Theoretical Neuroscience（雷德伍德理论神经科学中心）；University of California Berkeley（加州大学伯克利分校）；Berkeley AI Research（伯克利人工智能研究院）；Helen Wills Neuroscience Institute（海伦·威尔斯神经科学研究所）
Energy-Based Models (EBMs) assign unnormalized log-probability to data samples. This functionality has a variety of applications, such as sample synthesis, data denoising, sample restoration, outlier detection, Bayesian reasoning, and many more. But training of EBMs using standard maximum likelihood is extremely slow because it requires sampling from the model distribution. Score matching potentially alleviates this problem. In particular, denoising score matching \citep{vincent2011connection} has been successfully used to train EBMs. Using noisy data samples with one fixed noise level, these …
