The need for labeled data is among the most common and well-known practical obstacles to deploying deep learning algorithms to solve real-world problems. The current generation of learning algorithms requires a large volume of data labeled according to a static and pre-defined schema. Conversely, humans can quickly learn generalizations based on large quantities of unlabeled data, and turn these generalizations into classifications using spontaneous labels, often including labels not seen before. We apply a state-of-the-art unsupervised learning algorithm to the noisy and extremely imbalanced xView data set to train a feature extractor that adapts to several tasks: visual similarity search that performs well on both common and rare classes; identifying outliers within a labeled data set; and learning a natural class hierarchy automatically.
The potential of using remote sensing imagery for environmental modelling and for providing real time support to humanitarian operations such as hurricane relief efforts is well established. These applications are substantially affected by missing data due to non-structural noise such as clouds, shadows and other atmospheric effects. In this work we probe the potential of applying a cycle-consistent latent variable deep generative model (DGM) for denoising cloudy Sentinel-2 observations conditioned on the information in cloud penetrating bands. We adapt the recently proposed Fréchet Distance m…
Optical remote sensing imagery has been widely used in many fields due to its high resolution and stable geometric properties. However, remote sensing imagery is inevitably affected by climate, especially clouds. Removing the cloud in the high-resolution remote sensing satellite image is an indispensable pre-processing step before analyzing it. For the sake of large-scale training data, neural networks have been successful in many image processing tasks, but the use of neural networks to remove cloud in remote sensing imagery is still relatively small. We adopt generative adversarial network t…
School of Geosciences and Info Physics, Central South University（中南大学地球科学与信息物理学院）
Recently, deep convolutional neural network (DCNN) achieved increasingly remarkable success and rapidly developed in the field of natural image recognition. Compared with the natural image, the scale of remote sensing image is larger and the scene and the object it represents are more macroscopic. This study inquires whether remote sensing scene and natural scene recognitions differ and raises the following questions: What are the key factors in remote sensing scene recognition? Is the DCNN recognition mechanism centered on object recognition still applicable to the scenarios of remote sensing…
S\ a o Paulo State University - UNESP Department of Computing Bauru, SP - Brazil
Research on remote sensing image classification significantly impacts essential human routine tasks such as urban planning and agriculture. Nowadays, the rapid advance in technology and the availability of many high-quality remote sensing images create a demand for reliable automation methods. The current paper proposes two novel deep learning-based architectures for image classification purposes, i.e., the Discriminant Deep Image Prior Network and the Discriminant Deep Image Prior Network+, which combine Deep Image Prior and Triplet Networks learning strategies. Experiments conducted over thr…
Convolutional neural networks are currently the state-of-the-art algorithms for many remote sensing applications such as semantic segmentation or object detection. However, these algorithms are extremely sensitive to over-fitting, domain change and adversarial examples specifically designed to fool them. While adversarial attacks are not a threat in most remote sensing applications, one could wonder if strengthening networks to adversarial attacks could also increase their resilience to over-fitting and their ability to deal with the inherent variety of worldwide data. In this work, we study b…
Xi’an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences（中国科学院西安光机精密机械研究所）；The University of Chinese Academy of Sciences（中国科学院大学）
Inspired by recent development of artificial satellite, remote sensing images have attracted extensive attention. Recently, noticeable progress has been made in scene classification and target detection.However, it is still not clear how to describe the remote sensing image content with accurate and concise sentences. In this paper, we investigate to describe the remote sensing images with accurate and flexible sentences. First, some annotated instructions are presented to better describe the remote sensing images considering the special characteristics of remote sensing images. Second, in ord…
The past few decades have witnessed the great progress of unmanned aircraft vehicles (UAVs) in civilian fields, especially in photogrammetry and remote sensing. In contrast with the platforms of manned aircraft and satellite, the UAV platform holds many promising characteristics: flexibility, efficiency, high-spatial/temporal resolution, low cost, easy operation, etc., which make it an effective complement to other remote-sensing platforms and a cost-effective means for remote sensing. Considering the popularity and expansion of UAV-based remote sensing in recent years, this paper provides a s…
Nowadays government and private agencies use remote sensing imagery for a wide range of applications from military applications to farm development. The images may be a panchromatic, multispectral, hyperspectral or even ultraspectral of terra bytes. Remote sensing image classification is one amongst the most significant application worlds for remote sensing. A few number of image classification algorithms have proved good precision in classifying remote sensing data. But, of late, due to the increasing spatiotemporal dimensions of the remote sensing data, traditional classification algorithms …
Several remote sensing software packages are used to the explicit purpose of analyzing and visualizing remotely sensed data, with the developing of remote sensing sensor technologies from last ten years. Accord-ing to literature, the remote sensing is still the lack of software tools for effective information extraction from remote sensing data. So, this paper provides a state-of-art of multi-sensor image fusion technologies as well as review on the quality evaluation of the single image or fused images in the commercial remote sensing pack-ages. It also introduces program (ALwassaiProcess) de…
This paper introduces warped Gaussian processes (WGP) regression in remote sensing applications. WGP models output observations as a parametric nonlinear transformation of a GP. The parameters of such prior model are then learned via standard maximum likelihood. We show the good performance of the proposed model for the estimation of oceanic chlorophyll content from multispectral data, vegetation parameters (chlorophyll, leaf area index, and fractional vegetation cover) from hyperspectral data, and in the detection of the causal direction in a collection of 28 bivariate geoscience and remote s…
Given the importance of remote sensing, surprisingly little attention has been paid to it by the representation learning community. To address it and to establish baselines and a common evaluation protocol in this domain, we provide simplified access to 5 diverse remote sensing datasets in a standardized form. Specifically, we investigate in-domain representation learning to develop generic remote sensing representations and explore which characteristics are important for a dataset to be a good source for remote sensing representation learning. The established baselines achieve state-of-the-ar…
Shen Yuan Honors College, Beihang University（北航沈元荣誉学院）；Department of Aerospace Intelligent Science and Technology, School of Astronautics, Beihang University（北京航空航天大学宇航学院航天智能科学与技术系）；State Key Laboratory of Virtual Reality Technology and Systems, Beihang University（北京航空航天大学虚拟现实技术与系统国家重点实验室）；Qian Xuesen Laboratory of Space Technology, China Academy of Space Technology（中国空间技术研究院钱学森空间技术实验室）
Munich University of Applied Sciences（慕尼黑应用科学大学）；Technical University of Munich（慕尼黑工业大学）
Image classification is one of the main drivers of the rapid developments in deep learning with convolutional neural networks for computer vision. So is the analogous task of scene classification in remote sensing. However, in contrast to the computer vision community that has long been using well-established, large-scale standard datasets to train and benchmark high-capacity models, the remote sensing community still largely relies on relatively small and often application-dependend datasets, thus lacking comparability. With this letter, we present a classification-oriented conversion of the …
the University of Tokyo（东京大学）；RIKEN Center for Advanced Intelligence Project（理化学研究所先进智能项目中心）；Disaster Prevention Research Institute, Kyoto University（京都大学防灾研究所）；RIKEN Center for Computational Science（理化学研究所计算科学中心）；Hiroshima University（广岛大学）
We propose a framework that estimates inundation depth (maximum water level) and debris-flow-induced topographic deformation from remote sensing imagery by integrating deep learning and numerical simulation. A water and debris flow simulator generates training data for various artificial disaster scenarios. We show that regression models based on Attention U-Net and LinkNet architectures trained on such synthetic data can predict the maximum water level and topographic deformation from a remote sensing-derived change detection map and a digital elevation model. The proposed framework has an in…
Manifold Constraint Regularization for Remote Sensing Image Generation
Institute of Software Chinese Academy of Sciences, University of Chinese Academy of Sciences（中国科学院大学中国科学院软件研究所）；Tsinghua University（清华大学）；Hong Kong University of Science and Technology（香港科技大学）
本文首次提出流形约束正则化方法，通过新度量评估数据流形结构并约束 GAN 训练，缓解遥感图像生成中的过拟合问题，同时提升生成质量与下游任务性能。
Google Research India（谷歌印度研究院）；Google LLC（谷歌有限责任公司）
Self- and semi-supervised machine learning techniques leverage unlabeled data for improving downstream task performance. These methods are especially valuable for remote sensing tasks where producing labeled ground truth datasets can be prohibitively expensive but there is easy access to a wealth of unlabeled imagery. We perform a rigorous evaluation of SimCLR, a self-supervised method, and FixMatch, a semi-supervised method, on three remote sensing tasks: riverbed segmentation, land cover mapping, and flood mapping. We quantify performance improvements on these remote sensing segmentation tas…
Remote sensing change detection, identifying changes between scenes of the same location, is an active area of research with a broad range of applications. Recent advances in multimodal self-supervised pretraining have resulted in state-of-the-art methods which surpass vision models trained solely on optical imagery. In the remote sensing field, there is a wealth of overlapping 2D and 3D modalities which can be exploited to supervise representation learning in vision models. In this paper we propose Contrastive Surface-Image Pretraining (CSIP) for joint learning using optical RGB and above gro…
Recently, deep learning technology have been extensively used in the field of image recognition. However, its main application is the recognition and detection of ordinary pictures and common scenes. It is challenging to effectively and expediently analyze remote-sensing images obtained by the image acquisition systems on unmanned aerial vehicles (UAVs), which includes the identification of the target and calculation of its position. Aerial remote sensing images have different shooting angles and methods compared with ordinary pictures or images, which makes remote-sensing images play an irrep…
In this paper, a Multi-Scale Fully Convolutional Network (MSFCN) with multi-scale convolutional kernel is proposed to exploit discriminative representations from two-dimensional (2D) satellite images.
Research & Development Institute of Northwestern Polytechnical University in Shenzhen（西北工业大学深圳研究院）；School of Automation, Northwestern Polytechnical University（西北工业大学自动化学院）；School of Computer Science, Wuhan University（武汉大学计算机学院）
Remote sensing image scene classification, which aims at labeling remote sensing images with a set of semantic categories based on their contents, has broad applications in a range of fields. Propelled by the powerful feature learning capabilities of deep neural networks, remote sensing image scene classification driven by deep learning has drawn remarkable attention and achieved significant breakthroughs. However, to the best of our knowledge, a comprehensive review of recent achievements regarding deep learning for scene classification of remote sensing images is still lacking. Considering t…
Remote Sensing Technology Institute (IMF), German Aerospace Center (DLR)（德国航空航天中心遥感技术研究所）；Signal Processing in Earth Observation (SiPEO), Technical University of Munich (TUM)（慕尼黑工业大学地球观测信号处理实验室）；Department of Geography, University of Zurich（苏黎世大学地理系）；Laboratory of GeoInformation Science and Remote Sensing, Wageningen University of Research（瓦赫宁根大学地理信息科学与遥感实验室）；State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing (LIESMARS), Wuhan University（武汉大学测绘遥感信息工程国家重点实验室）；Institute of Computer Graphics and Vision, TU Graz（格拉茨技术大学计算机图形与视觉研究所）
Standing at the paradigm shift towards data-intensive science, machine learning techniques are becoming increasingly important. In particular, as a major breakthrough in the field, deep learning has proven as an extremely powerful tool in many fields. Shall we embrace deep learning as the key to all? Or, should we resist a 'black-box' solution? There are controversial opinions in the remote sensing community. In this article, we analyze the challenges of using deep learning for remote sensing data analysis, review the recent advances, and provide resources to make deep learning in remote sensi…
Generic Knowledge Boosted Pre-training For Remote Sensing Images
State Key Laboratory of Virtual Reality Technology and Systems, Beihang University（北京航空航天大学虚拟现实技术与系统国家重点实验室）；Hangzhou Innovation Institute, Beihang University（北京航空航天大学杭州创新研究院）；School of Software and Microelectronics, Peking University（北京大学软件与微电子学院）
University of Central Florida（中佛罗里达大学）；UT Arlington（德克萨斯大学阿灵顿分校）；NIT Delhi（德里国立理工学院）
Remote sensing has become a crucial part of our daily lives, whether it be from triangulating our location using GPS or providing us with a weather forecast. It has multiple applications in domains such as military, socio-economical, commercial, and even in supporting humanitarian efforts. This work proposes a novel technique for the automated geo-location of Oil & Gas infrastructure with the use of Active Fire Data from the NASA FIRMS data repository & Deep Learning techniques; achieving a top accuracy of 90.68% with the use of ResNet101.
Recent advances in computer vision using deep learning with RGB imagery (e.g., object recognition and detection) have been made possible thanks to the development of large annotated RGB image datasets. In contrast, multispectral image (MSI) and hyperspectral image (HSI) datasets contain far fewer labeled images, in part due to the wide variety of sensors used. These annotations are especially limited for semantic segmentation, or pixel-wise classification, of remote sensing imagery because it is labor intensive to generate image annotations. Low-shot learning algorithms can make effective infe…
Diverse Instance Generation via Diffusion Models for Enhanced Few-Shot Object Detection in Remote Sensing Images
Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Tsinghua University（清华大学）
MBZ University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Wuhan University（武汉大学）
Deep learning-based algorithms have seen a massive popularity in different areas of remote sensing image analysis over the past decade. Recently, transformers-based architectures, originally introduced in natural language processing, have pervaded computer vision field where the self-attention mechanism has been utilized as a replacement to the popular convolution operator for capturing long-range dependencies. Inspired by recent advances in computer vision, remote sensing community has also witnessed an increased exploration of vision transformers for a diverse set of tasks. Although a number…
Automatically finding good and general remote sensing representations allows to perform transfer learning on a wide range of applications - improving the accuracy and reducing the required number of training samples. This paper investigates development of generic remote sensing representations, and explores which characteristics are important for a dataset to be a good source for representation learning. For this analysis, five diverse remote sensing datasets are selected and used for both, disjoint upstream representation learning and downstream model training and evaluation. A common evaluat…
A combinatory approach of two well-known fields: deep learning and semi supervised learning is presented, to tackle the land cover identification problem. The proposed methodology demonstrates the impact on the performance of deep learning models, when SSL approaches are used as performance functions during training. Obtained results, at pixel level segmentation tasks over orthoimages, suggest that SSL enhanced loss functions can be beneficial in models' performance.
Beijing Institute of Technology（北京理工大学）；Mississippi State University（密西西比州立大学）
Multi-source image registration is challenging due to intensity, rotation, and scale differences among the images. Considering the characteristics and differences of multi-source remote sensing images, a feature-based registration algorithm named Multi-scale Histogram of Local Main Orientation (MS-HLMO) is proposed. Harris corner detection is first adopted to generate feature points. The HLMO feature of each Harris feature point is extracted on a Partial Main Orientation Map (PMOM) with a Generalized Gradient Location and Orientation Histogram-like (GGLOH) feature descriptor, which provides hi…
Skolkovo institute of Science and technology（斯科尔科沃科学技术研究所）
Recent developments in the remote sensing systems and image processing made it possible to propose a new method of the object classification and detection of the specific changes in the series of satellite Earth images (so called targeted change detection). In this paper we propose a formal problem statement that allows to use effectively the deep learning approach to analyze time-dependent series of remote sensing images. We also introduce a new framework for the development of deep learning models for targeted change detection and demonstrate some cases of business applications it can be use…
Beijing Forestry University（北京林业大学）；Tsinghua University（清华大学）；Chinese Institute for Brain Research (CIBR)（中国脑科学研究中心（CIBR））
Extracting cultivated land accurately from high-resolution remote images is a basic task for precision agriculture. This report introduces our solution to the iFLYTEK challenge 2021 cultivated land extraction from high-resolution remote sensing image. The challenge requires segmenting cultivated land objects in very high-resolution multispectral remote sensing images. We established a highly effective and efficient pipeline to solve this problem. We first divided the original images into small tiles and separately performed instance segmentation on each tile. We explored several instance segme…
With the development of remote sensing technology, the acquisition of remote sensing images is easier and easier, which provides sufficient data resources for the task of detecting remote sensing objects. However, how to detect objects quickly and accurately from many complex optical remote sensing images is a challenging hot issue. In this paper, we propose an efficient anchor free object detector, CenterFPANet. To pursue speed, we use a lightweight backbone and introduce the asymmetric revolution block. To improve the accuracy, we designed the FPA module, which links the feature maps of diff…
Remote sensing images and techniques are powerful tools to investigate earth surface. Data quality is the key to enhance remote sensing applications and obtaining a clear and noise-free set of data is very difficult in most situations due to the varying acquisition (e.g., atmosphere and season), sensor, and platform (e.g., satellite angles and sensor characteristics) conditions. With the increasing development of satellites, nowadays Terabytes of remote sensing images can be acquired every day. Therefore, information and data fusion can be particularly important in the remote sensing community…
Element AI；Universitat Politècnica de Catalunya（加泰罗尼亚理工大学）
Remote sensing and automatic earth monitoring are key to solve global-scale challenges such as disaster prevention, land use monitoring, or tackling climate change. Although there exist vast amounts of remote sensing data, most of it remains unlabeled and thus inaccessible for supervised learning algorithms. Transfer learning approaches can reduce the data requirements of deep learning algorithms. However, most of these methods are pre-trained on ImageNet and their generalization to remote sensing imagery is not guaranteed due to the domain gap. In this work, we propose Seasonal Contrast (SeCo…
Institute of Electronics, Chinese Academy of Sciences（中国科学院电子学研究所）
Cloud-based overlays are often present in optical remote sensing images, thus limiting the application of acquired data. Removing clouds is an indispensable pre-processing step in remote sensing image analysis. Deep learning has achieved great success in the field of remote sensing in recent years, including scene classification and change detection. However, deep learning is rarely applied in remote sensing image removal clouds. The reason is the lack of data sets for training neural networks. In order to solve this problem, this paper first proposed the Remote sensing Image Cloud rEmoving da…
Entropy is the measure of uncertainty in any data and is adopted for maximisation of mutual information in many remote sensing operations. The availability of wide entropy variations motivated us for an investigation over the suitability preference of these versions to specific operations.
Carlson Center for Imaging Science, Rochester Institute of Technology（卡尔森成像科学中心，罗切斯特理工学院）
Deep learning continues to push state-of-the-art performance for the semantic segmentation of color (i.e., RGB) imagery; however, the lack of annotated data for many remote sensing sensors (i.e. hyperspectral imagery (HSI)) prevents researchers from taking advantage of this recent success. Since generating sensor specific datasets is time intensive and cost prohibitive, remote sensing researchers have embraced deep unsupervised feature extraction. Although these methods have pushed state-of-the-art performance on current HSI benchmarks, many of these tools are not readily accessible to many re…
The proliferation of remote sensing satellites has resulted in a massive amount of remote sensing images. However, due to human and material resource constraints, the vast majority of remote sensing images remain unlabeled. As a result, it cannot be applied to currently available deep learning methods. To fully utilize the remaining unlabeled images, we propose a Geographical Knowledge-driven Representation learning method for remote sensing images (GeoKR), improving network performance and reduce the demand for annotated data. The global land cover products and geographical location associate…
Remote sensing semantic segmentation aims to assign automatically each pixel on aerial images with specific label. In this letter, we proposed a new module, called improved-flow warp module (IFWM), to adjust semantic feature maps across different scales for remote sensing semantic segmentation. The improved-flow warp module is applied along with the feature extraction process in the convolutional neural network. First, IFWM computes the offsets of pixels by a learnable way, which can alleviate the misalignment of the multi-scale features. Second, the offsets help with the low-resolution deep f…
University of Tehran（德黑兰大学）；Amirkabir University of Technology (Tehran Polytechnic)（阿米尔卡比尔理工大学（德黑兰理工学院））；University of Sydney（悉尼大学）；University of New South Wales（新南威尔士大学）
The decline of the number of newly discovered mineral deposits and increase in demand for different minerals in recent years has led exploration geologists to look for more efficient and innovative methods for processing different data types at each stage of mineral exploration. As a primary step, various features, such as lithological units, alteration types, structures, and indicator minerals, are mapped to aid decision-making in targeting ore deposits. Different types of remote sensing datasets, such as satellite and airborne data, make it possible to overcome common problems associated wit…
Thanks to recent advances in CNNs, solid improvements have been made in semantic segmentation of high resolution remote sensing imagery. However, most of the previous works have not fully taken into account the specific difficulties that exist in remote sensing tasks. One of such difficulties is that objects are small and crowded in remote sensing imagery. To tackle with this challenging task we have proposed a novel architecture called local feature extraction (LFE) module attached on top of dilated front-end module. The LFE module is based on our findings that aggressively increasing dilatio…
In recent years, supervised learning has been widely used in various tasks of optical remote sensing image understanding, including remote sensing image classification, pixel-wise segmentation, change detection, and object detection. The methods based on supervised learning need a large amount of high-quality training data and their performance highly depends on the quality of the labels. However, in practical remote sensing applications, it is often expensive and time-consuming to obtain large-scale data sets with high-quality labels, which leads to a lack of sufficient supervised information…
