We propose a novel approach for generating high quality visible-like images from Synthetic Aperture Radar (SAR) images using Deep Convolutional Generative Adversarial Network (GAN) architectures. The proposed approach is based on a cascaded network of convolutional neural nets (CNNs) for despeckling and image colorization. The cascaded structure results in faster convergence during training and produces high quality visible images from the corresponding SAR images. Experimental results on both simulated and real SAR images show that the proposed method can produce visible-like images better compared to the recent state-of-the-art deep learning-based methods.
Toshiba Infrastructure Systems & Solutions Corporation（东芝基础设施系统与解决方案公司）
This report deals with translation invariance of convolutional neural networks (CNNs) for automatic target recognition (ATR) from synthetic aperture radar (SAR) imagery. In particular, the translation invariance of CNNs for SAR ATR represents the robustness against misalignment of target chips extracted from SAR images. To understand the translation invariance of the CNNs, we trained CNNs which classify the target chips from the MSTAR into the ten classes under the condition of with and without data augmentation, and then visualized the translation invariance of the CNNs. According to our resu…
University of Chinese Academy of Sciences（中国科学院大学）；Chinese Academy of Sciences（中国科学院）；Institute of Electronics, Chinese Academy of Sciences（中国科学院电子学研究所）
Deep convolutional neural networks (DCNNs) have attracted much attention in remote sensing recently. Compared with the large-scale annotated dataset in natural images, the lack of labeled data in remote sensing becomes an obstacle to train a deep network very well, especially in SAR image interpretation. Transfer learning provides an effective way to solve this problem by borrowing the knowledge from the source task to the target task. In optical remote sensing application, a prevalent mechanism is to fine-tune on an existing model pre-trained with a large-scale natural image dataset, such as …
Nagoya University（名古屋大学）；Chubu University（中部大学）；Tokyo Institute of Technology（东京工业大学）；Advanced Industrial Science and Technology（产业技术综合研究所）
In this paper, we propose a method for cloud removal from visible light RGB satellite images by extending the conditional Generative Adversarial Networks (cGANs) from RGB images to multispectral images. Satellite images have been widely utilized for various purposes, such as natural environment monitoring (pollution, forest or rivers), transportation improvement and prompt emergency response to disasters. However, the obscurity caused by clouds makes it unstable to monitor the situation on the ground with the visible light camera. Images captured by a longer wavelength are introduced to reduce…
University of Florence（佛罗伦萨大学）；Centre Tecnològic Telecomunicacions Catalunya（加泰罗尼亚电信技术中心）；University of Padua（帕多瓦大学）
In this letter, we use deep-learning convolution neural networks (CNNs) to assess the landslide mapping and classification performances on optical images (from Sentinel-2) and SAR images (from Sentinel-1). The training and test zones used to independently evaluate the performance of the CNNs on different datasets are located in the eastern Iburi subprefecture in Hokkaido, where, at 03.08 local time (JST) on September 6, 2018, an Mw 6.6 earthquake triggered about 8000 coseismic landslides. We analyzed the conditions before and after the earthquake exploiting multi-polarization SAR as well as op…
Convolutional neural networks (CNNs) have been applied to learn spatial features for high-resolution (HR) synthetic aperture radar (SAR) image classification. However, there has been little work on integrating the unique statistical distributions of SAR images which can reveal physical properties of terrain objects, into CNNs in a supervised feature learning framework. To address this problem, a novel end-to-end supervised classification method is proposed for HR SAR images by considering both spatial context and statistical features. First, to extract more effective spatial features from SAR …
Université Paris Est（巴黎东大学）；CNRS（法国国家科学研究中心）；ESIEE Paris（巴黎高等电子与电工技术工程师学院）；University Federico II of Naples（那不勒斯费德里科二世大学）
In this paper we investigate the use of discriminative model learning through Convolutional Neural Networks (CNNs) for SAR image despeckling. The network uses a residual learning strategy, hence it does not recover the filtered image, but the speckle component, which is then subtracted from the noisy one. Training is carried out by considering a large multitemporal SAR image and its multilook version, in order to approximate a clean image. Experimental results, both on synthetic and real SAR data, show the method to achieve better performance with respect to state-of-the-art techniques.
With the booming of Convolutional Neural Networks (CNNs), CNNs such as VGG-16 and ResNet-50 widely serve as backbone in SAR ship detection. However, CNN based backbone is hard to model long-range dependencies, and causes the lack of enough high-quality semantic information in feature maps of shallow layers, which leads to poor detection performance in complicated background and small-sized ships cases. To address these problems, we propose a SAR ship detection method based on Swin Transformer and Feature Enhancement Feature Pyramid Network (FEFPN). Swin Transformer serves as backbone to model …
Synthetic Aperture Radar (SAR) despeckling is an important problem in remote sensing as speckle degrades SAR images, affecting downstream tasks like detection and segmentation. Recent studies show that convolutional neural networks(CNNs) outperform classical despeckling methods. Traditional CNNs try to increase the receptive field size as the network goes deeper, thus extracting global features. However,speckle is relatively small, and increasing receptive field does not help in extracting speckle features. This study employs an overcomplete CNN architecture to focus on learning low-level feat…
Multispectral disparity estimation is a difficult task for many reasons: it has all the same challenges as traditional visible-visible disparity estimation (occlusions, repetitive patterns, textureless surfaces), in addition of having very few common visual information between images (e.g. color information vs. thermal information). In this paper, we propose a new CNN architecture able to do disparity estimation between images from different spectrum, namely thermal and visible in our case. Our proposed model takes two patches as input and proceeds to do domain feature extraction for each of t…
University Parthenope（帕特诺佩大学）；University Federico II（费德里科二世大学）
We propose a new method for SAR image despeckling which leverages information drawn from co-registered optical imagery. Filtering is performed by plain patch-wise nonlocal means, operating exclusively on SAR data. However, the filtering weights are computed by taking into account also the optical guide, which is much cleaner than the SAR data, and hence more discriminative. To avoid injecting optical-domain information into the filtered image, a SAR-domain statistical test is preliminarily performed to reject right away any risky predictor. Experiments on two SAR-optical datasets prove the pro…
We propose a new convolutional neural network (CNN) which performs coarse and fine segmentation for end-to-end synthetic aperture radar (SAR) automatic target recognition (ATR) system. In recent years, many CNNs for SAR ATR using deep learning have been proposed, but most of them classify target classes from fixed size target chips extracted from SAR imagery. On the other hand, we proposed the CNN which outputs the score of the multiple target classes and a background class for each pixel from the SAR imagery of arbitrary size and multiple targets as fine segmentation. However, it was necessar…
Toshiba Infrastructure Systems & Solutions Corporation（东芝基础设施系统与解决方案公司）
The standard architecture of synthetic aperture radar (SAR) automatic target recognition (ATR) consists of three stages: detection, discrimination, and classification. In recent years, convolutional neural networks (CNNs) for SAR ATR have been proposed, but most of them classify target classes from a target chip extracted from SAR imagery, as a classification for the third stage of SAR ATR. In this report, we propose a novel CNN for end-to-end ATR from SAR imagery. The CNN named verification support network (VersNet) performs all three stages of SAR ATR end-to-end. VersNet inputs a SAR image o…
A machine learning approach for image classification in synthetic aperture RADAR
Department of Mathematics and Statistics University of Limerick（数学与统计学系利默里克大学）；Health Research Institute (HRI)（健康研究中心）；Department of Computer Science and Information Systems University of Limerick（计算机科学与信息系统系利默里克大学）
Synthetic Aperture Radar (SAR) constitutes a fundamental asset for wide-areas monitoring with high-resolution requirements. The first SAR sensors have given rise to coarse coastal and maritime monitoring applications, including oil spill, ship and ice floes detection. With the upgrade to very high-resolution sensors in the recent years, with relatively new SAR missions such as Sentinel-1, a great deal of data providing a stronger information content has been released, enabling more refined studies on general targets features and thus permitting complex classifications, as for ship classificati…
Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Suzhou Research Institute, Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院苏州研究院）
At present, the Synthetic Aperture Radar (SAR) image classification method based on convolution neural network (CNN) has faced some problems such as poor noise resistance and generalization ability. Spiking neural network (SNN) is one of the core components of brain-like intelligence and has good application prospects. This article constructs a complete SAR image classifier based on unsupervised and supervised learning of SNN by using spike sequences with complex spatio-temporal information. We firstly expound the spiking neuron model, the receptive field of SNN, and the construction of spike …
Qatar University（卡塔尔大学）；Universiti Kebangsaan Malaysia（马来西亚国民大学）
Novel Coronavirus disease (COVID-19) is an extremely contagious and quickly spreading Coronavirus infestation. Severe Acute Respiratory Syndrome (SARS) and Middle East Respiratory Syndrome (MERS), which outbreak in 2002 and 2011, and the current COVID-19 pandemic are all from the same family of coronavirus. This work aims to classify COVID-19, SARS, and MERS chest X-ray (CXR) images using deep Convolutional Neural Networks (CNNs). A unique database was created, so-called QU-COVID-family, consisting of 423 COVID-19, 144 MERS, and 134 SARS CXR images. Besides, a robust COVID-19 recognition syste…
Ocean University of China（中国海洋大学）；Southwest Jiaotong University（西南交通大学）
Traditional synthetic aperture radar image change detection methods based on convolutional neural networks (CNNs) face the challenges of speckle noise and deformation sensitivity. To mitigate these issues, we proposed a Multiscale Capsule Network (Ms-CapsNet) to extract the discriminative information between the changed and unchanged pixels. On the one hand, the multiscale capsule module is employed to exploit the spatial relationship of features. Therefore, equivariant properties can be achieved by aggregating the features from different positions. On the other hand, an adaptive fusion convol…
Multi-Expert Learning Framework with the State Space Model for Optical and SAR Image Registration
Key Laboratory of Intelligent Perception and Image Understanding of Ministry of Education of China（教育部智能感知与图像理解重点实验室）；Hangzhou Institute of Technology, Xidian University（西安电子科技大学杭州学院）；School of Computer Science, Xidian University（西安电子科技大学计算机学院）；Department of Automation, Tsinghua University（清华大学自动化系）
Ocean University of China（中国海洋大学）；Southwest Jiaotong University（西南交通大学）
Traditional change detection methods based on convolutional neural networks (CNNs) face the challenges of speckle noise and deformation sensitivity for synthetic aperture radar images. To mitigate these issues, we proposed a Multiscale Capsule Network (Ms-CapsNet) to extract the discriminative information between the changed and unchanged pixels. On the one hand, the capsule module is employed to exploit the spatial relationship of features. Therefore, equivariant properties can be achieved by aggregating the features from different positions. On the other hand, an adaptive fusion convolution …
Synthetic Aperture Radar (SAR) images are prone to be contaminated by noise, which makes it very difficult to perform target recognition in SAR images. Inspired by great success of very deep convolutional neural networks (CNNs), this paper proposes a robust feature extraction method for SAR image target classification by adaptively fusing effective features from different CNN layers. First, YOLOv4 network is fine-tuned to detect the targets from the respective MF SAR target images. Second, a very deep CNN is trained from scratch on the moving and stationary target acquisition and recognition (…
Optical image data have been used by the Remote Sensing workforce to study land use and cover since such data is easily interpretable. Synthetic Aperture Radar (SAR) has the characteristic of obtaining images during all-day, all-weather and provides object information that is different from visible and infrared sensors. However, SAR images have more speckle noise and fewer dimensions. This paper presents a method for denoising, feature extraction and compares classifications of Optical and SAR images. The image was denoised using K-Singular Value Decomposition (K-SVD) algorithm. A method to ma…
SARS-CoV-2, also known as COVID-19 or Coronavirus, is a viral contagious disease that is infected by a novel coronavirus, and has been rapidly spreading across the globe. It is very important to test and isolate people to reduce spread, and from here comes the need to do this quickly and efficiently. According to some studies, Chest-CT outperforms RT-PCR lab testing, which is the current standard, when diagnosing COVID-19 patients. Due to this, computer vision researchers have developed various deep learning systems that can predict COVID-19 using a Chest-CT scan correctly to a certain degree.…
Speckle reduction is a longstanding topic in synthetic aperture radar (SAR) images. Many different schemes have been proposed for the restoration of intensity SAR images. Among the different possible approaches, methods based on convolutional neural networks (CNNs) have recently shown to reach state-of-the-art performance for SAR image restoration. CNN training requires good training data: many pairs of speckle-free / speckle-corrupted images. This is an issue in SAR applications, given the inherent scarcity of speckle-free images. To handle this problem, this paper analyzes different strategi…
Enhancing Robustness of Human Detection Algorithms in Maritime SAR through Augmented Aerial Images to Simulate Weather Conditions
Jakarta Intercultural School（雅加达国际学校）；Busan Foreign School（釜山外国语学校）；SMPK 8 Penabur Tanjung Duren（丹戎杜伦佩纳布尔8号初中（基督教学校））；Avon High School（雅芳高中）；Algoverse AI Research（Algoverse人工智能研究院）
Semantic inpainting is the task of inferring missing pixels in an image given surrounding pixels and high level image semantics. Most semantic inpainting algorithms are deterministic: given an image with missing regions, a single inpainted image is generated. However, there are often several plausible inpaintings for a given missing region. In this paper, we propose a method to perform probabilistic semantic inpainting by building a model, based on PixelCNNs, that learns a distribution of images conditioned on a subset of visible pixels. Experiments on the MNIST and CelebA datasets show that o…
International Institute of Information Technology Naya Raipur（国际信息技术学院纳亚赖普尔分校）
Skin cancer is one of the most prevalent forms of human cancer. It is recognized mainly visually, beginning with clinical screening and continuing with the dermoscopic examination, histological assessment, and specimen collection. Deep convolutional neural networks (CNNs) perform highly segregated and potentially universal tasks against a classified finegrained object. This research proposes a novel multi-class prediction framework that classifies skin lesions based on ViT and ViTGAN. Vision transformers-based GANs (Generative Adversarial Networks) are utilized to tackle the class imbalance. T…
Remote Sensing Technology Institute, German Aerospace Center（德国航空航天中心遥感技术研究所）；Signal Processing in Earth Observation, Technical University of Munich（慕尼黑工业大学地球观测信号处理部门）
Object retrieval and reconstruction from very high resolution (VHR) synthetic aperture radar (SAR) images are of great importance for urban SAR applications, yet highly challenging owing to the complexity of SAR data. This paper addresses the issue of individual building segmentation from a single VHR SAR image in large-scale urban areas. To achieve this, we introduce building footprints from GIS data as complementary information and propose a novel conditional GIS-aware network (CG-Net). The proposed model learns multi-level visual features and employs building footprints to normalize the fea…
In this work, we exploit convolutional neural networks (CNNs) for the classification of very high resolution (VHR) polarimetric SAR (PolSAR) data. Due to the significant appearance of heterogeneous textures within these data, not only polarimetric features but also structural tensors are exploited to feed CNN models. For deep networks, we use the SegNet model for semantic segmentation, which corresponds to pixelwise classification in remote sensing. Our experiments on the airborne F-SAR data show that for VHR PolSAR images, SegNet could provide high accuracy for the classification task; and in…
Maharaja Agrasen Institute of Technology（马哈拉贾·阿格拉森技术学院）；Near East University（近东大学）；State University of Ceará（塞阿拉州立大学）；University of Fortaleza（福塔莱萨大学）
Coronavirus (COVID-19) is a viral disease caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). The spread of COVID-19 seems to have a detrimental effect on the global economy and health. A positive chest X-ray of infected patients is a crucial step in the battle against COVID-19. Early results suggest that abnormalities exist in chest X-rays of patients suggestive of COVID-19. This has led to the introduction of a variety of deep learning systems and studies have shown that the accuracy of COVID-19 patient detection through the use of chest X-rays is strongly optimistic. Dee…
Cross-modal learning for SAR target recognition using optical vision foundation models
School of Engineering, University of Edinburgh（爱丁堡大学工程学院）；Leonardo UK（英国莱昂纳多公司）；School of Engineering and Physical Sciences, Heriot-Watt University（赫瑞瓦特大学工程与物理科学学院）
Synthetic Aperture Radar Image Change Detection Based on Global Dynamic Context-Aware Network
State Key Laboratory of Physical Oceanography, Ocean University of China（中国海洋大学物理海洋国家重点实验室）；Department of Electrical and Computer Engineering, Mississippi State University（密苏里州立大学电气与计算机工程系）
State Key Laboratory of Media Convergence and Communication, Communication University of China（媒体融合与传播国家重点实验室，中国传媒大学）；Neuroscience and Intelligent Media Institute, Communication University of China（神经科学与智能媒体研究院，中国传媒大学）
The partially occluded image recognition (POIR) problem has been a challenge for artificial intelligence for a long time. A common strategy to handle the POIR problem is using the non-occluded features for classification. Unfortunately, this strategy will lose effectiveness when the image is severely occluded, since the visible parts can only provide limited information. Several studies in neuroscience reveal that feature restoration which fills in the occluded information and is called amodal completion is essential for human brains to recognize partially occluded images. However, feature res…
Nankai University（南开大学）；Advanced Digital Sciences Center（先进数字科学中心）；Huazhong University of Science and Technology（华中科技大学）；Nanjing University of Science and Technology（南京理工大学）
In this paper, we propose an accurate edge detector using richer convolutional features (RCF). Since objects in nature images have various scales and aspect ratios, the automatically learned rich hierarchical representations by CNNs are very critical and effective to detect edges and object boundaries. And the convolutional features gradually become coarser with receptive fields increasing. Based on these observations, our proposed network architecture makes full use of multiscale and multi-level information to perform the image-to-image edge prediction by combining all of the useful convoluti…
Institute of Information Systems, HES-SO (University of Applied Sciences and Arts Western Switzerland)（瑞士西部应用科学与艺术大学 信息系统研究所）；Department of Computer Science, University of Geneva（日内瓦大学 计算机科学系）；Department of Neurosciences, University of Padua（帕多瓦大学 神经科学系）；Inselspital, Bern Hospital（伯尔尼大学医院）；Medical Faculty, University of Geneva（日内瓦大学 医学院）
Computational pathology is a domain that aims to develop algorithms to automatically analyze large digitized histopathology images, called whole slide images (WSI). WSIs are produced scanning thin tissue samples that are stained to make specific structures visible. They show stain colour heterogeneity due to different preparation and scanning settings applied across medical centers. Stain colour heterogeneity is a problem to train convolutional neural networks (CNN), the state-of-the-art algorithms for most computational pathology tasks, since CNNs usually underperform when tested on images in…
Object Detection is critical for automatic military operations. However, the performance of current object detection algorithms is deficient in terms of the requirements in military scenarios. This is mainly because the object presence is hard to detect due to the indistinguishable appearance and dramatic changes of object's size which is determined by the distance to the detection sensors. Recent advances in deep learning have achieved promising results in many challenging tasks. The state-of-the-art in object detection is represented by convolutional neural networks (CNNs), such as the fast …
Obstacle detection is a fundamental capability of an autonomous maritime surface vessel (AMSV). State-of-the-art obstacle detection algorithms are based on convolutional neural networks (CNNs). While CNNs provide higher detection accuracy and fast detection speed, they require enormous amounts of data for their training. In particular, the availability of domain-specific datasets is a challenge for obstacle detection. The difficulty in conducting onsite experiments limits the collection of maritime datasets. Owing to the logistic cost of conducting on-site operations, simulation tools provide …
City, University of London（伦敦大学城市学院）；European Space Agency（欧洲空间局）
Recent interest in on-orbit servicing and Active Debris Removal (ADR) missions have driven the need for technologies to enable non-cooperative rendezvous manoeuvres. Such manoeuvres put heavy burden on the perception capabilities of a chaser spacecraft. This paper demonstrates Convolutional Neural Networks (CNNs) capable of providing an initial coarse pose estimation of a target from a passive thermal infrared camera feed. Thermal cameras offer a promising alternative to visible cameras, which struggle in low light conditions and are susceptible to overexposure. Often, thermal information on t…
