Sea-land segmentation is an important process for many key applications in remote sensing. Proper operative sea-land segmentation for remote sensing images remains a challenging issue due to complex and diverse transition between sea and lands. Although several Convolutional Neural Networks (CNNs) have been developed for sea-land segmentation, the performance of these CNNs is far from the expected target. This paper presents a novel deep neural network structure for pixel-wise sea-land segmentation, a Residual Dense U-Net (RDU-Net), in complex and high-density remote sensing images. RDU-Net is a combination of both down-sampling and up-sampling paths to achieve satisfactory results. In each down- and up-sampling path, in addition to the convolution layers, several densely connected residual network blocks are proposed to systematically aggregate multi-scale contextual information. Each dense network block contains multilevel convolution layers, short-range connections and an identity mapping connection which facilitates features re-use in the network and makes full use of the hierarchical features from the original images. These proposed blocks have a certain number of connections that are designed with shorter distance backpropagation between the layers and can significantly improve segmentation results whilst minimizing computational costs. We have performed extensive experiments on two real datasets Google Earth and ISPRS and compare the proposed RDUNet against several variations of Dense Networks. The experimental results show that RDUNet outperforms the other state-of-the-art approaches on the sea-land segmentation tasks.
Land-cover classification using remote sensing imagery is an important Earth observation task. Recently, land cover classification has benefited from the development of fully connected neural networks for semantic segmentation. The benchmark datasets available for training deep segmentation models in remote sensing imagery tend to be small, however, often consisting of only a handful of images from a single location with a single scale. This limits the models' ability to generalize to other datasets. Domain adaptation has been proposed to improve the models' generalization but we find these ap…
Wuhan University（武汉大学）；Chinese Academy of Sciences（中国科学院）；China Land Survey and Planning Institute（中国国土勘测规划院）
In recent years, large amount of high spatial-resolution remote sensing (HRRS) images are available for land-cover mapping. However, due to the complex information brought by the increased spatial resolution and the data disturbances caused by different conditions of image acquisition, it is often difficult to find an efficient method for achieving accurate land-cover classification with high-resolution and heterogeneous remote sensing images. In this paper, we propose a scheme to apply deep model obtained from labeled land-cover dataset to classify unlabeled HRRS images. The main idea is to r…
Land-cover classification has long been a hot and difficult challenge in remote sensing community. With massive High-resolution Remote Sensing (HRS) images available, manually and automatically designed Convolutional Neural Networks (CNNs) have already shown their great latent capacity on HRS land-cover classification in recent years. Especially, the former can achieve better performance while the latter is able to generate lightweight architecture. Unfortunately, they both have shortcomings. On the one hand, because manual CNNs are almost proposed for natural image processing, it becomes very…
Remote Sensing Technology Institute, German Aerospace Center（德国航空航天中心遥感技术研究所）；Technical University of Munich（慕尼黑工业大学）；Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）；Univ. Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；INRIA（法国国家信息与自动化研究所）；CNRS（法国国家科学研究中心）；Grenoble INP（格勒诺布尔理工学院）
As remote sensing (RS) data obtained from different sensors become available largely and openly, multimodal data processing and analysis techniques have been garnering increasing interest in the RS and geoscience community. However, due to the gap between different modalities in terms of imaging sensors, resolutions, and contents, embedding their complementary information into a consistent, compact, accurate, and discriminative representation, to a great extent, remains challenging. To this end, we propose a shared and specific feature learning (S2FL) model. S2FL is capable of decomposing mult…
University of Kentucky（肯塔基大学）；Indiana University Bloomington（印第安纳大学伯明顿分校）
We propose a novel convolutional neural network architecture for estimating geospatial functions such as population density, land cover, or land use. In our approach, we combine overhead and ground-level images in an end-to-end trainable neural network, which uses kernel regression and density estimation to convert features extracted from the ground-level images into a dense feature map. The output of this network is a dense estimate of the geospatial function in the form of a pixel-level labeling of the overhead image. To evaluate our approach, we created a large dataset of overhead and groun…
Institute of Advanced Research in Artificial Intelligence（人工智能高级研究院）；Helmholtz-Zentrum Dresden-Rossendorf（亥姆霍兹德累斯顿罗森多夫研究中心）；Helmholtz Institute Freiberg for Resource Technology（亥姆霍兹弗赖贝格资源技术研究所）
This study introduces \textit{Landslide4Sense}, a reference benchmark for landslide detection from remote sensing. The repository features 3,799 image patches fusing optical layers from Sentinel-2 sensors with the digital elevation model and slope layer derived from ALOS PALSAR. The added topographical information facilitates the accurate detection of landslide borders, which recent researches have shown to be challenging using optical data alone. The extensive data set supports deep learning (DL) studies in landslide detection and the development and validation of methods for the systematic u…
MMLSv2: A Multimodal Dataset for Martian Landslide Detection in Remote Sensing Imagery
Fairfield University（费尔菲尔德大学）；Michigan Technological University（密歇根技术大学）；Computer Vision Center（计算机视觉中心）；Universitat Autònoma de Barcelona（巴塞罗那自治大学）；ESPOL Polytechnic University（ESPOL多科大学）；University of Kerala（喀拉拉大学）；University of Mississippi（密西西比大学）
Center for Machine Vision and Signal Analysis, University of Oulu（奥卢大学机器视觉与信号分析中心）；Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所智能信息处理重点实验室）；University of Chinese Academy of Sciences（中国科学院大学）
Remote photoplethysmography (rPPG), which aims at measuring heart activities without any contact, has great potential in many applications (e.g., remote healthcare). Existing end-to-end rPPG and heart rate (HR) measurement methods from facial videos are vulnerable to the less-constrained scenarios (e.g., with head movement and bad illumination). In this letter, we explore the reason why existing end-to-end networks perform poorly in challenging conditions and establish a strong end-to-end baseline (AutoHR) for remote HR measurement with neural architecture search (NAS). The proposed method inc…
Recently, a massive number of deep learning based approaches have been successfully applied to various remote sensing image (RSI) recognition tasks. However, most existing advances of deep learning methods in the RSI field heavily rely on the features extracted by the manually designed backbone network, which severely hinders the potential of deep learning models due the complexity of RSI and the limitation of prior knowledge. In this paper, we research a new design paradigm for the backbone architecture in RSI recognition tasks, including scene classification, land-cover classification and ob…
Changes to Captions: An Attentive Network for Remote Sensing Change Captioning
Institute of Advanced Research in Artificial Intelligence（人工智能高级研究院）；Helmholtz-Zentrum Dresden-Rossendorf (HZDR)（亥姆霍兹德累斯顿罗森多夫研究中心）；Helmholtz Institute Freiberg for Resource Technology (HIF)（亥姆霍兹弗赖贝格资源技术研究所）
This paper presents the BigEarthNet that is a new large-scale multi-label Sentinel-2 benchmark archive. The BigEarthNet consists of 590,326 Sentinel-2 image patches, each of which is a section of i) 120x120 pixels for 10m bands; ii) 60x60 pixels for 20m bands; and iii) 20x20 pixels for 60m bands. Unlike most of the existing archives, each image patch is annotated by multiple land-cover classes (i.e., multi-labels) that are provided from the CORINE Land Cover database of the year 2018 (CLC 2018). The BigEarthNet is significantly larger than the existing archives in remote sensing (RS) and thus …
Deep learning has become in recent years a cornerstone tool fueling key innovations in the industry, such as autonomous driving. To attain good performances, the neural network architecture used for a given application must be chosen with care. These architectures are often handcrafted and therefore prone to human biases and sub-optimal selection. Neural Architecture Search (NAS) is a framework introduced to mitigate such risks by jointly optimizing the network architectures and its weights. Albeit its novelty, it was applied on complex tasks with significant results - e.g. semantic image segm…
GeoChrono: Benchmarking and Rethinking Long-Term Temporal Understanding in Remote Sensing
Beijing University of Posts and Telecommunications（北京邮电大学）；Tsinghua University（清华大学）；Hunan Normal University（湖南师范大学）；City University of Hong Kong（香港城市大学）
The Ohio State University（俄亥俄州立大学）；Michigan Technological University（密歇根理工大学）
As an important application in remote sensing, landcover classification remains one of the most challenging tasks in very-high-resolution (VHR) image analysis. As the rapidly increasing number of Deep Learning (DL) based landcover methods and training strategies are claimed to be the state-of-the-art, the already fragmented technical landscape of landcover mapping methods has been further complicated. Although there exists a plethora of literature review work attempting to guide researchers in making an informed choice of landcover mapping methods, the articles either focus on the review of ap…
In this paper, a Multi-Scale Fully Convolutional Network (MSFCN) with multi-scale convolutional kernel is proposed to exploit discriminative representations from two-dimensional (2D) satellite images.
SRM TRP Engineering College（SRM TRP工程学院）；University of Sannio（萨尼奥大学）；GSSS Institute of Engineering and Technology for Women（GSSS女子工程技术学院）
Land use and land cover (LULC) classification using remote sensing imagery plays a vital role in many environment modeling and land use inventories. In this study, a hybrid feature optimization algorithm along with a deep learning classifier is proposed to improve the performance of LULC classification, helping to predict wildlife habitat, deteriorating environmental quality, haphazard, etc. LULC classification is assessed using Sat 4, Sat 6 and Eurosat datasets. After the selection of remote sensing images, normalization and histogram equalization methods are used to improve the quality of th…
Centre for Research-based Innovation Visual Intelligence（可视化研究型创新中心）
Multi-modality data is becoming readily available in remote sensing (RS) and can provide complementary information about the Earth's surface. Effective fusion of multi-modal information is thus important for various applications in RS, but also very challenging due to large domain differences, noise, and redundancies. There is a lack of effective and scalable fusion techniques for bridging multiple modality encoders and fully exploiting complementary information. To this end, we propose a new multi-modality network (MultiModNet) for land cover mapping of multi-modal remote sensing data based o…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Detecting and masking cloud and cloud shadow from satellite remote sensing images is a pervasive problem in the remote sensing community. Accurate and efficient detection of cloud and cloud shadow is an essential step to harness the value of remotely sensed data for almost all downstream analysis. DeepMask, a new algorithm for cloud and cloud shadow detection in optical satellite remote sensing imagery, is proposed in this study. DeepMask utilizes ResNet, a deep convolutional neural network, for pixel-level cloud mask generation. The algorithm is trained and evaluated on the Landsat 8 Cloud Co…
RMAU-NET: A Residual-Multihead-Attention U-Net Architecture for Landslide Segmentation and Detection from Remote Sensing Images
Van Lang University（范兰大学）；Austrian Institute of Technology（奥地利技术研究院）；Troyes University（特鲁瓦大学）；HCM University of Technology（胡志明技术大学）；Ton Duc Thang University（-ton Duc Thang大学）
本文提出了一种基于残差多头注意力 U-Net 架构的端到端深度学习模型 RMAU-NET，利用遥感图像进行滑坡检测与分割，在多个基准数据集上取得了优异的 F1 和 mIoU 分数，证明了其在实际滑坡观测系统中的应用潜力。
University of Campinas（坎皮纳斯州立大学）；Wageningen University & Research（瓦赫宁根大学及研究中心）
OpenStreetMap (OSM) is a community-based, freely available, editable map service that was created as an alternative to authoritative ones. Given that it is edited mainly by volunteers with different mapping skills, the completeness and quality of its annotations are heterogeneous across different geographical locations. Despite that, OSM has been widely used in several applications in {Geosciences}, Earth Observation and environmental sciences. In this work, we present a review of recent methods based on machine learning to improve and use OSM data. Such methods aim either 1) at improving the …
Satellite remote imaging enables the detailed study of land use patterns on a global scale. We investigate the possibility to improve the information content of traditional land use classification by identifying the nature of industrial sites from medium-resolution remote sensing images. In this work, we focus on classifying different types of power plants from Sentinel-2 imaging data. Using a ResNet-50 deep learning model, we are able to achieve a mean accuracy of 90.0% in distinguishing 10 different power plant types and a background class. Furthermore, we are able to identify the cooling me…
Landslide Detection and Segmentation Using Remote Sensing Images and Deep Neural Network
Austrian Institute of Technology（奥地利技术研究院）；GeoSphere Austria（奥地利地球物理、气象和地球物理研究所）；University of Natural Resources and Life Sciences, Vienna（维也纳自然资源与生命科学大学）
With the advancement of remote-sensed imaging large volumes of very high resolution land cover images can now be obtained. Automation of object recognition in these 2D images, however, is still a key issue. High intra-class variance and low inter-class variance in Very High Resolution (VHR) images hamper the accuracy of prediction in object recognition tasks. Most successful techniques in various computer vision tasks recently are based on deep supervised learning. In this work, a deep Convolutional Neural Network (CNN) based on symmetric encoder-decoder architecture with skip connections is e…
Adaptive Frequency Enhancement Network for Remote Sensing Image Semantic Segmentation
School of Computer Science and Technology, Ocean University of China（中国海洋大学计算机科学与技术学院）；Department of Electrical and Computer Engineering, Mississippi State University（密西西比州立大学电气与计算机工程系）
Ocean University of China（中国海洋大学）；Mississippi State University（密西西比州立大学）
Existing remote sensing change detection methods are heavily affected by seasonal variation. Since vegetation colors are different between winter and summer, such variations are inclined to be falsely detected as changes. In this letter, we proposed an image translation method to solve the problem. A style-based recalibration module is introduced to capture seasonal features effectively. Then, a new style discriminator is designed to improve the translation performance. The discriminator can not only produce a decision for the fake or real sample, but also return a style vector according to th…
Estimating forest carbon stocks from high-resolution remote sensing imagery by reducing domain shift with style transfer
School of Geography and Remote Sensing, Guangzhou University（广州大学地理与遥感学院）；Innovation Center for Remote Sensing Big Data Intelligent Applications, Guangzhou University（广州大学遥感大数据智能应用创新中心）
With the acceleration of the urban expansion, urban change detection (UCD), as a significant and effective approach, can provide the change information with respect to geospatial objects for dynamical urban analysis. However, existing datasets suffer from three bottlenecks: (1) lack of high spatial resolution images; (2) lack of semantic annotation; (3) lack of long-range multi-temporal images. In this paper, we propose a large scale benchmark dataset, termed Hi-UCD. This dataset uses aerial images with a spatial resolution of 0.1 m provided by the Estonia Land Board, including three-time phas…
Undirected graphical models have been successfully used to jointly model the spatial and the spectral dependencies in earth observing hyperspectral images. They produce less noisy, smooth, and spatially coherent land cover maps and give top accuracies on many datasets. Moreover, they can easily be combined with other state-of-the-art approaches, such as deep learning. This has made them an essential tool for remote sensing researchers and practitioners. However, graphical models have not been easily accessible to the larger remote sensing community as they are not discussed in standard remote …
Text-RSIR: A Text-Guided Framework for Efficient Remote Sensing Image Transmission and Reconstruction
School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）科学与工程学院）；Geosciences and Engineering, Southwest Jiaotong University（西南交通大学地球科学与工程学院）；Institute of Space and Earth Information Science and the Department of Geography and Resource Management, The Chinese University of Hong Kong（香港中文大学空间与地球信息科学研究所及地理与资源管理系）
Wuhan University（武汉大学）；German Aerospace Center (DLR)（德国航空航天中心）；Technical University of Munich（慕尼黑工业大学）
Targeting at depicting land covers with pixel-wise semantic categories, semantic segmentation in remote sensing images needs to portray diverse distributions over vast geographical locations, which is difficult to be achieved by the homogeneous pixel-wise forward paths in the architectures of existing deep models. Although several algorithms have been designed to select pixel-wise adaptive forward paths for natural image analysis, it still lacks theoretical supports on how to obtain optimal selections. In this paper, we provide mathematical analyses in terms of the parameter optimization, whic…
Miyun Reservoir has produced huge benefits in flood control, agricultural irrigation, power generation, aquaculture, tourism, and urban water supply. Accurately water mapping is of great significance to the ecological environment monitoring of the Miyun Reservoir and the management of the South-to-North Water Diversion Project. On the 60th anniversary of the completion of the Miyun Reservoir, we took the Miyun Reservoir as the study area and collected all the Landsat-5 and Landsat-8 remote sensing images from 1984 to 2020 for water mapping. Based on the spectral, topographical and temporal-spa…
Change Detection Methods for Remote Sensing in the Last Decade: A Comprehensive Review
University of Liverpool（利物浦大学）；Beihang University（北京航空航天大学）；Cambridge University（剑桥大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
In this paper a new Geophysical gravimetry approach is presented, which is based on satellite imagery in remote sensing. The method uses a satellite image, together with a set of points in the image the gravity values of which are known. Template-based spheroidal spline method of interpolation is used to constitute a system of equations to find the values of gravity at other points in the image. A real case study is presented for the Qom region in Iran. Values of gravity are determined from the Landsat satellite image for this region, using 9 points in the image whose gravity values are known.…
