Polarimetric SAR Image Semantic Segmentation with 3D Discrete Wavelet Transform and Markov Random Field
University of Bristol（布里斯托大学）；Shanghai Em-Data Technology Co., Ltd.（上海数慧科技股份有限公司）；China University of Mining and Technology（中国矿业大学）；University of Derby（德比大学）
Haixia Bi, Lin Xu, Xiangyong Cao, Yong Xue, Zongben Xu
Polarimetric synthetic aperture radar (PolSAR) image segmentation is currently of great importance in image processing for remote sensing applications. However, it is a challenging task due to two main reasons. Firstly, the label information is difficult to acquire due to high annotation costs. Secondly, the speckle effect embedded in the PolSAR imaging process remarkably degrades the segmentation performance. To address these two issues, we present a contextual PolSAR image semantic segmentation method in this paper.With a newly defined channelwise consistent feature set as input, the three-dimensional discrete wavelet transform (3D-DWT) technique is employed to extract discriminative multi-scale features that are robust to speckle noise. Then Markov random field (MRF) is further applied to enforce label smoothness spatially during segmentation. By simultaneously utilizing 3D-DWT features and MRF priors for the first time, contextual information is fully integrated during the segmentation to ensure accurate and smooth segmentation. To demonstrate the effectiveness of the proposed method, we conduct extensive experiments on three real benchmark PolSAR image data sets. Experimental results indicate that the proposed method achieves promising segmentation accuracy and preferable spatial consistency using a minimal number of labeled pixels.
Instituto de Computação, Universidade Federal de Alagoas（阿拉戈斯联邦大学计算研究所）；Departamento de Computación, Universidad de Buenos Aires（布宜诺斯艾利斯大学计算系）
We present an approach for polarimetric Synthetic Aperture Radar (SAR) image region boundary detection based on the use of B-Spline active contours and a new model for polarimetric SAR data: the GHP distribution. In order to detect the boundary of a region, initial B-Spline curves are specified, either automatically or manually, and the proposed algorithm uses a deformable contours technique to find the boundary. In doing this, the parameters of the polarimetric GHP model for the data are estimated, in order to find the transition points between the region being segmented and the surrounding a…
Key Lab for Information Science of Electromagnetic Waves (MoE), Fudan University（复旦大学电磁波信息科学教育部重点实验室）
A deep neural networks based method is proposed to convert single polarization grayscale SAR image to fully polarimetric. It consists of two components: a feature extractor network to extract hierarchical multi-scale spatial features of grayscale SAR image, followed by a feature translator network to map spatial feature to polarimetric feature with which the polarimetric covariance matrix of each pixel can be reconstructed. Both qualitative and quantitative experiments with real fully polarimetric data are conducted to show the efficacy of the proposed method. The reconstructed full-pol SAR im…
The data fusion technology aims to aggregate the characteristics of different data and obtain products with multiple data advantages. To solves the problem of reduced resolution of PolSAR images due to system limitations, we propose a fully polarimetric synthetic aperture radar (PolSAR) images and single-polarization synthetic aperture radar SAR (SinSAR) images fusion network to generate high-resolution PolSAR (HR-PolSAR) images. To take advantage of the polarimetric information of the low-resolution PolSAR (LR-PolSAR) image and the spatial information of the high-resolution single-polarizatio…
Prior-guided Fusion of Multimodal Features for Change Detection from Optical-SAR Images
Institute of Geospatial Information, Information Engineering University（地理信息研究所，信息工程大学）；Academy of Digital China (Fujian), Fuzhou University（数字中国研究院（福建），福州大学）；The School of Electronics and Communication Engineering, Sun Yat-sen University（电子与通信工程学院，中山大学）；The Department of Information Engineering and Computer Science, University of Trento（信息工程与计算机科学系，特伦托大学）
In this paper, we introduce an iterative speckle filtering method for polarimetric SAR (PolSAR) images based on the bilateral filter. To locally adapt to the spatial structure of images, this filter relies on pixel similarities in both spatial and radiometric domains. To deal with polarimetric data, we study the use of similarities based on a statistical distance called Kullback-Leibler divergence as well as two geodesic distances on Riemannian manifolds. To cope with speckle, we propose to progressively refine the result thanks to an iterative scheme. Experiments are run over synthetic and ex…
SAOCOM Mission, National Commission of Space Activities (CONAE)（阿根廷国家空间活动委员会 SAOCOM 任务）
This paper describes a novel projection algorithm, the Projection Onto Span Algorithm (POSA) for wavelet-based superresolution and removing speckle (in wavelet domain) of unknown variance from Synthetic Aperture Radar (SAR) images. Although the POSA is good as a new superresolution algorithm for image enhancement, image metrology and biometric identification, here one will use it like a tool of despeckling, being the first time that an algorithm of super-resolution is used for despeckling of SAR images. Specifically, the speckled SAR image is decomposed into wavelet subbands, POSA is applied t…
Weakly-supervised land classification for coastal zone based on deep convolutional neural networks by incorporating dual-polarimetric characteristics into training dataset
The wavelet shrinkage denoising approach is able to maintain local regularity of a signal while suppressing noise. However, the conventional wavelet shrinkage based methods are not time-scale adaptive to track the local time-scale variation. In this paper, a new type of Neural Shrinkage (NS) is presented with a new class of shrinkage architecture for speckle reduction in Synthetic Aperture Radar (SAR) images. The numerical results indicate that the new method outperforms the standard filters, the standard wavelet shrinkage despeckling method, and previous NS.
In this paper, a new probability density function (pdf) is proposed to model the statistics of wavelet coefficients, and a simple Kalman's filter is derived from the new pdf using Bayesian estimation theory. Specifically, we decompose the speckled image into wavelet subbands, we apply the Kalman's filter to the high subbands, and reconstruct a despeckled image from the modified detail coefficients. Experimental results demonstrate that our method compares favorably to several other despeckling methods on test synthetic aperture radar (SAR) images.
Bridging Modalities and Tasks: A Unified Hierarchical ViT for SAR-to-Optical Translation and Semantic Segmentation
School of Automation, Northwestern Polytechnical University（西北工业大学自动化学院）；School of Cybersecurity, Northwestern Polytechnical University（西北工业大学网络空间安全学院）
We describe a novel method for removing speckle (in wavelet domain) of unknown variance from SAR images. The me-thod is based on the following procedure: We apply 1) Bidimentional Discrete Wavelet Transform (DWT-2D) to the speckled image, 2) scaling and rounding to the coefficients of the highest subbands (to obtain integer and positive coefficients), 3) bit-slicing to the new highest subbands (to obtain bit-planes), 4) then we apply the Systholic Boolean Orthonormalizer Network (SBON) to the input bit-plane set and we obtain two orthonormal output bit-plane sets (in a Boolean sense), we proje…
Remote Sensing Technology Institute (IMF), German Aerospace Center (DLR)（德国航空航天中心（DLR）遥感技术研究所（IMF））；Signal Processing in Earth Observation (SiPEO), Technical University of Munich (TUM)（慕尼黑工业大学（TUM）地球观测信号处理实验室（SiPEO））
Multi-modal data fusion has recently been shown promise in classification tasks in remote sensing. Optical data and radar data, two important yet intrinsically different data sources, are attracting more and more attention for potential data fusion. It is already widely known that, a machine learning based methodology often yields excellent performance. However, the methodology relies on a large training set, which is very expensive to achieve in remote sensing. The semi-supervised manifold alignment (SSMA), a multi-modal data fusion algorithm, has been designed to amplify the impact of an exi…
Traditional survey methods for finding surface resistivity are time-consuming and labor intensive. Very few studies have focused on finding the resistivity/conductivity using remote sensing data and deep learning techniques. In this line of work, we assessed the correlation between surface resistivity and Synthetic Aperture Radar (SAR) by applying various deep learning methods and tested our hypothesis in the Coso Geothermal Area, USA. For detecting the resistivity, L-band full polarimetric SAR data acquired by UAVSAR were used, and MT (Magnetotellurics) inverted resistivity data of the area w…
School of Microelectronics and Communication Engineering, Chongqing University（重庆大学微电子与通信工程学院）；Chongqing Key Laboratory of Space Information Network and Intelligent Information Fusion（重庆市空间信息网络与智能信息融合重点实验室）；Lancaster Environment Centre, Lancaster University（兰卡斯特大学兰卡斯特环境中心）；UK Centre for Ecology & Hydrology（英国生态与水文中心）；Department of Computer Science, Aberystwyth University（阿伯里斯特威斯大学计算机科学系）；Geography and Environmental Science, University of Southampton（南安普顿大学地理与环境科学系）；Institute of Geographic Sciences and Natural Resources Research, Chinese Academy of Sciences（中国科学院地理科学与资源研究所）
Small area change detection from synthetic aperture radar (SAR) is a highly challenging task. In this paper, a robust unsupervised approach is proposed for small area change detection from multi-temporal SAR images using deep learning. First, a multi-scale superpixel reconstruction method is developed to generate a difference image (DI), which can suppress the speckle noise effectively and enhance edges by exploiting local, spatially homogeneous information. Second, a two-stage centre-constrained fuzzy c-means clustering algorithm is proposed to divide the pixels of the DI into changed, unchan…
Polarimetric synthetic aperture radar (PolSAR) images are widely used in disaster detection and military reconnaissance and so on. However, their interpretation faces some challenges, e.g., deficiency of labeled data, inadequate utilization of data information and so on. In this paper, a complex-valued generative adversarial network (GAN) is proposed for the first time to address these issues. The complex number form of model complies with the physical mechanism of PolSAR data and in favor of utilizing and retaining amplitude and phase information of PolSAR data. GAN architecture and semi-supe…
Avinashilingam University for Women（阿维纳什林加姆女子大学）；Naval Research Board-DRDO（海军研究委员会-国防研究与发展组织）
A new fangled method for ship wake detection in synthetic aperture radar (SAR) images is explored here. Most of the detection procedure applies the Radon transform as its properties outfit more than any other transformation for the detection purpose. But still it holds problems when the transform is applied to an image with a high level of noise. Here this paper articulates the combination between the radon transformation and the shrinkage methods which increase the mode of wake detection process. The latter shrinkage method with RT maximize the signal to noise ratio hence it leads to most opt…
Riemannian Complex Hermit Positive Definite Convolution Network for Polarimetric SAR Image Classification
Department of Computer Science and Technology, Shaanxi Key Laboratory for Network Computing and Security Technology, Xi’an University of Technology（计算机科学与技术系、网络计算与安全技术陕西省重点实验室、西安理工大学）；School of Artificial Intelligence, Xidian University（人工智能学院、西安电子科技大学）
As a representation learning method, nearest regularized subspace(NRS) algorithm is an effective tool to obtain both accuracy and speed for PolSAR image classification. However, existing NRS methods use the polarimetric feature vector but the PolSAR original covariance matrix(known as Hermitian positive definite(HPD)matrix) as the input. Without considering the matrix structure, existing NRS-based methods cannot learn correlation among channels. How to utilize the original covariance matrix to NRS method is a key problem. To address this limit, a Riemannian NRS method is proposed, which consid…
Automatized marine vessel monitoring from sentinel-1 data using convolution neural network
Center for Environment & Water, Research Institute, King Fahd University of Petroleum & Minerals（法赫德国王石油与矿产大学环境与水研究中心）；University of Petroleum and Energy Studies（石油与能源研究大学）；Dr. B.C. Roy Engineering College（B.C.罗伊博士工程学院）；CubicX
针对传统 CFAR 船舶检测缺乏决策能力的问题，提出基于小波变换的卷积神经网络，利用 Sentinel-1 双极化 SAR 数据实现印度西海岸繁忙交通场景下的船舶自动识别，检测准确率达 95.46%。
This work deals with unsupervised image deblurring. We present a new deblurring procedure on images provided by low-resolution synthetic aperture radar (SAR) or simply by multimedia in presence of multiplicative (speckle) or additive noise, respectively. The method we propose is defined as a two-step process. First, we use an original technique for noise reduction in wavelet domain. Then, the learning of a Kohonen self-organizing map (SOM) is performed directly on the denoised image to take out it the blur. This technique has been successfully applied to real SAR images, and the simulation res…
Instituto Nacional de Pesquisas Espaciais – INPE（巴西国家空间研究院）
This paper presents a technique for reducing speckle in Polarimetric Synthetic Aperture Radar (PolSAR) imagery using Nonlocal Means and a statistical test based on stochastic divergences. The main objective is to select homogeneous pixels in the filtering area through statistical tests between distributions. This proposal uses the complex Wishart model to describe PolSAR data, but the technique can be extended to other models. The weights of the location-variant linear filter are function of the p-values of tests which verify the hypothesis that two samples come from the same distribution and,…
Polarimetric Synthetic Aperture Radar (PolSAR) images are establishing as an important source of information in remote sensing applications. The most complete format this type of imaging produces consists of complex-valued Hermitian matrices in every image coordinate and, as such, their visualization is challenging. They also suffer from speckle noise which reduces the signal-to-noise ratio. Smoothing techniques have been proposed in the literature aiming at preserving different features and, analogously, projections from the cone of Hermitian positive matrices to different color representatio…
University of Trento（特伦托大学）；Information Engineering University（信息工程大学）；Space Engineering University NCO School（航天工程大学士官学校）
There are limited studies on the semantic segmentation of high-resolution Polarimetric Synthetic Aperture Radar (PolSAR) images due to the scarcity of training data and the inference of speckle noises. The Gaofen contest has provided open access of a high-quality PolSAR semantic segmentation dataset. Taking this chance, we propose a Multi-path ResNet (MP-ResNet) architecture for the semantic segmentation of high-resolution PolSAR images. Compared to conventional U-shape encoder-decoder convolutional neural network (CNN) architectures, the MP-ResNet learns semantic context with its parallel mul…
In this work, we exploit convolutional neural networks (CNNs) for the classification of very high resolution (VHR) polarimetric SAR (PolSAR) data. Due to the significant appearance of heterogeneous textures within these data, not only polarimetric features but also structural tensors are exploited to feed CNN models. For deep networks, we use the SegNet model for semantic segmentation, which corresponds to pixelwise classification in remote sensing. Our experiments on the airborne F-SAR data show that for VHR PolSAR images, SegNet could provide high accuracy for the classification task; and in…
For polarimetric SAR (PolSAR) image classification, it is a challenge to classify the aggregated terrain types, such as the urban area, into semantic homogenous regions due to sharp bright-dark variations in intensity. The aggregated terrain type is formulated by the similar ground objects aggregated together. In this paper, a polarimetric hierarchical semantic model (PHSM) is firstly proposed to overcome this disadvantage based on the constructions of a primal-level and a middle-level semantic. The primal-level semantic is a polarimetric sketch map which consists of sketch segments as the spa…
Polarimetric SAR data has the characteristics of all-weather, all-time and so on, which is widely used in many fields. However, the data of annotation is relatively small, which is not conducive to our research. In this paper, we have collected five open polarimetric SAR images, which are images of the San Francisco area. These five images come from different satellites at different times, which has great scientific research value. We annotate the collected images at the pixel level for image classification and segmentation. For the convenience of researchers, the annotated data is open source…
Facultad de Ingeniería, Universidad de Buenos Aires（布宜诺斯艾利斯大学工程学院）；Universidad Nacional de Tierra del Fuego, Antártida e Islas del Atlántico Sur（火地岛、南极和南大西洋岛屿国立大学）；Instituto Argentino de Matemática ”A. P. Calderón”, IAM, CONICET（阿根廷数学研究所“AP卡尔德隆”（IAM），阿根廷国家科学技术研究委员会（CONICET））；Instituto de Ciencias Polares, Ambiente y Recursos Naturales（极地、环境和自然资源研究所）
In this note we deal with the detection of oil spills in open sea via self similar, long range dependence random fields and wavelet filters. We show some preliminary experimental results of our technique with Sentinel 1 SAR images.
The application of wavelet transforms to Synthetic Aperture Radar (SAR) imagery has improved despeckling performance. To deduce the problem of filtering the multiplicative noise to the case of an additive noise, the wavelet decomposition is performed on the logarithm of the image gray levels. The detail coefficients produced by the bidimensional discrete wavelet transform (DWT-2D) needs to be thresholded to extract out the speckle in highest subbands. An initial threshold value is estimated according to the noise variance. In this paper, an additional fuzzy thresholding approach for automatic …
University of Florida（佛罗里达大学）；Miami University（迈阿密大学）；Air Force Research Laboratory（空军研究实验室）
Deep learning-based image retrieval has been emphasized in computer vision. Representation embedding extracted by deep neural networks (DNNs) not only aims at containing semantic information of the image, but also can manage large-scale image retrieval tasks. In this work, we propose a deep learning-based image retrieval approach using homography transformation augmented contrastive learning to perform large-scale synthetic aperture radar (SAR) image search tasks. Moreover, we propose a training method for the DNNs induced by contrastive learning that does not require any labeling procedure. T…
PolSAM: Polarimetric Scattering Mechanism Informed Segment Anything Model
the BRain and Artificial INtelligence Lab (BRAIN LAB), School of Automation, Northwestern Polytechnical University（人工智能实验室（BRAIN LAB）、自动化学院、西北工业大学）；the Shenzhen Research Institute of Northwestern Polytechnical University（西北工业大学深圳研究院）；the National Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University（多媒体信息处理国家重点实验室、计算机科学学院、北京大学）；the National Key Laboratory of Microwave Imaging Technology, Chinese Academy of Sciences（微波成像技术国家重点实验室、中国科学院）；the Aerospace Information Research Institute, Chinese Academy of Sciences（航空信息研究所、中国科学院）
Convolutional neural networks (CNNs) have been applied to learn spatial features for high-resolution (HR) synthetic aperture radar (SAR) image classification. However, there has been little work on integrating the unique statistical distributions of SAR images which can reveal physical properties of terrain objects, into CNNs in a supervised feature learning framework. To address this problem, a novel end-to-end supervised classification method is proposed for HR SAR images by considering both spatial context and statistical features. First, to extract more effective spatial features from SAR …
Single-pixel imaging is an indirect imaging technique which utilizes simplified optical hardware and advanced computational methods. It offers novel solutions for hyper-spectral imaging, polarimetric imaging, three-dimensional imaging, holographic imaging, optical encryption and imaging through scattering media. The main limitations for its use come from relatively high measurement and reconstruction times. In this paper we propose to reduce the required signal acquisition time by using a novel sampling scheme based on a random selection of Morlet wavelets convolved with white noise. While suc…
Automatic Target Recognition (ATR) in Synthetic aperture radar (SAR) images becomes a very challenging problem owing to containing high level noise. In this study, a machine learning-based method is proposed to detect different moving and stationary targets using SAR images. First Order Statistical (FOS) features were obtained from Fast Fourier Transform (FFT), Discrete Cosine Transform (DCT) and Discrete Wavelet Transform (DWT) on gray level SAR images. Gray Level Co-occurrence Matrix (GLCM), Gray Level Run Length Matrix (GLRLM) and Gray Level Size Zone Matrix (GLSZM) algorithms are also used…
The recognition or understanding of the scenes observed with a SAR system requires a broader range of cues, beyond the spatial context. These encompass but are not limited to: imaging geometry, imaging mode, properties of the Fourier spectrum of the images or the behavior of the polarimetric signatures. In this paper, we propose a change of paradigm for explainability in data science for the case of Synthetic Aperture Radar (SAR) data to ground the explainable AI for SAR. It aims to use explainable data transformations based on well-established models to generate inputs for AI methods, to prov…
Instituto Nacional de Pesquisas Espaciais（国家空间研究所）；Universidade Federal de Alagoas（阿拉戈斯联邦大学）；Universidad Pública de Navarra（纳瓦拉公立大学）
We address the issue of adapting optical images-based edge detection techniques for use in Polarimetric Synthetic Aperture Radar (PolSAR) imagery. We modify the gravitational edge detection technique (inspired by the Law of Universal Gravity) proposed by Lopez-Molina et al, using the non-standard neighbourhood configuration proposed by Fu et al, to reduce the speckle noise in polarimetric SAR imagery. We compare the modified and unmodified versions of the gravitational edge detection technique with the well-established one proposed by Canny, as well as with a recent multiscale fuzzy-based tech…
Beijing University of Chemical Technology（北京化工大学）；Sugon Information Industry Co., Ltd（曙光信息产业有限公司）
The land cover classification has played an important role in remote sensing because it can intelligently identify things in one huge remote sensing image to reduce the work of humans. However, a lot of classification methods are designed based on the pixel feature or limited spatial feature of the remote sensing image, which limits the classification accuracy and universality of their methods. This paper proposed a novel method to take into the information of remote sensing image, i.e., geographic latitude-longitude information. In addition, a dual-branch convolutional neural network (CNN) cl…
M3LEO: A Multi-Modal, Multi-Label Earth Observation Dataset Integrating Interferometric SAR and Multispectral Data
University of Cambridge（剑桥大学）；Drexel University（德雷塞尔大学）；Universitat de València（瓦伦西亚大学）；European Space Agency（欧洲空间局）；University of Oxford（牛津大学）；Universidad de Antioquia（安蒂奥基亚大学）
Radar (SAR) images often exhibit profound appearance variations due to a variety of factors including clutter noise produced by the coherent nature of the illumination. Ultrasound images and infrared images have similar cluttered appearance, that make 1 dimensional structures, as edges and object boundaries difficult to locate. Structure information is usually extracted in two steps: first, building and edge strength mask classifying pixels as edge points by hypothesis testing, and secondly estimating from that mask, pixel wide connected edges. With constant false alarm rate (CFAR) edge streng…
Efficient and accurate polarimetric synthetic aperture radar (PolSAR) image classification with a limited number of prior labels is always full of challenges. For general supervised deep learning classification algorithms, the pixel-by-pixel algorithm achieves precise yet inefficient classification with a small number of labeled pixels, whereas the pixel mapping algorithm achieves efficient yet edge-rough classification with more prior labels required. To take efficiency, accuracy and prior labels into account, we propose a novel pixel-refining parallel mapping network in the complex domain na…
