The production of thematic maps depicting land cover is one of the most common applications of remote sensing. To this end, several semantic segmentation approaches, based on deep learning, have been proposed in the literature, but land cover segmentation is still considered an open problem due to some specific problems related to remote sensing imaging. In this paper we propose a novel approach to deal with the problem of modelling multiscale contexts surrounding pixels of different land cover categories. The approach leverages the computation of a heteroscedastic measure of uncertainty when classifying individual pixels in an image. This classification uncertainty measure is used to define a set of memory gates between layers that allow a principled method to select the optimal decision for each pixel.
University of Basel（巴塞尔大学）；University Hospital Basel（巴塞尔大学医院）；MIAC AG
We present an automated method for localizing an anatomical landmark in three-dimensional medical images. The method combines two recurrent neural networks in a coarse-to-fine approach: The first network determines a candidate neighborhood by analyzing the complete given image volume. The second network localizes the actual landmark precisely and accurately in the candidate neighborhood. Both networks take advantage of multi-dimensional gated recurrent units in their main layers, which allow for high model complexity with a comparatively small set of parameters. We localize the medullopontine …
Henry M. Gunn High School（亨利·M·冈恩高中）；Rutgers University（罗格斯大学）
This paper presents a change detection method that identifies land cover changes from aerial imagery, using semantic segmentation, a machine learning approach. We present a land cover classification training pipeline with Deeplab v3+, state-of-the-art semantic segmentation technology, including data preparation, model training for seven land cover types, and model exporting modules. In the land cover change detection system, the inputs are images retrieved from Google Earth at the same location but from different times. The system then predicts semantic segmentation results on these images usi…
Many significant applications need land cover information of remote sensing images that are acquired from different areas and times, such as change detection and disaster monitoring. However, it is difficult to find a generic land cover classification scheme for different remote sensing images due to the spectral shift caused by diverse acquisition condition. In this paper, we develop a novel land cover classification method that can deal with large-scale data captured from widely distributed areas and different times. Additionally, we establish a large-scale land cover classification dataset …
Harbin Institute of Technology（哈尔滨工业大学）；SenseTime Group Limited（商汤科技有限公司）
Recently, FCNs based methods have made great progress in semantic segmentation. Different with ordinary scenes, satellite image owns specific characteristics, which elements always extend to large scope and no regular or clear boundaries. Therefore, effective mid-level structure information extremely missing, precise pixel-level classification becomes tough issues. In this paper, a Dense Fusion Classmate Network (DFCNet) is proposed to adopt in land cover classification.
Deep learning approaches have shown promising results in remote sensing high spatial resolution (HSR) land-cover mapping. However, urban and rural scenes can show completely different geographical landscapes, and the inadequate generalizability of these algorithms hinders city-level or national-level mapping. Most of the existing HSR land-cover datasets mainly promote the research of learning semantic representation, thereby ignoring the model transferability. In this paper, we introduce the Land-cOVEr Domain Adaptive semantic segmentation (LoveDA) dataset to advance semantic and transferable …
Learning with less: label-efficient land cover classification at very high spatial resolution using self-supervised deep learning
Department of Agricultural and Biological Engineering, Mississippi State University（农业与生物工程系，密苏里州立大学）；North Mississippi Research and Extension Center, Mississippi State University（北密苏里研究与推广中心，密苏里州立大学）
Technical University of Munich（慕尼黑工业大学）；German Aerospace Center (DLR)（德国航空航天中心（DLR））
Fully automatic large-scale land cover mapping belongs to the core challenges addressed by the remote sensing community. Usually, the basis of this task is formed by (supervised) machine learning models. However, in spite of recent growth in the availability of satellite observations, accurate training data remains comparably scarce. On the other hand, numerous global land cover products exist and can be accessed often free-of-charge. Unfortunately, these maps are typically of a much lower resolution than modern day satellite imagery. Besides, they always come with a significant amount of nois…
A large-scale climate-aware satellite image dataset for domain adaptive land-cover semantic segmentation
School of Remote Sensing Information Engineering, Wuhan University（武汉大学遥感信息工程学院）；State Key Laboratory of Geo-information Engineering（地理信息工程国家重点实验室）；Advanced Research Institute of Multidisciplinary Science, Beijing Institute of Technology（北京理工大学多学科先进研究院）；School of Computer Science and Technology, Beijing Institute of Technology（北京理工大学计算机学院）；School of Electronics and Communication Engineering, Sun Yat-sen University（中山大学电子与通信工程学院）
Tallinn University of Technology（塔林理工大学）；University of Oxford（牛津大学）
The focus of this paper is using a convolutional machine learning model with a modified U-Net structure for creating land cover classification mapping based on satellite imagery. The aim of the research is to train and test convolutional models for automatic land cover mapping and to assess their usability in increasing land cover mapping accuracy and change detection. To solve these tasks, authors prepared a dataset and trained machine learning models for land cover classification and semantic segmentation from satellite images. The results were analysed on three different land classification…
In the remote sensing community, Land Use Land Cover (LULC) classification with satellite imagery is a main focus of current research activities. Accurate and appropriate LULC classification, however, continues to be a challenging task. In this paper, we evaluate the performance of multi-temporal (monthly time series) compared to mono-temporal (single time step) satellite images for multi-label classification using supervised learning on the RapidAI4EO dataset. As a first step, we trained our CNN model on images at a single time step for multi-label classification, i.e. mono-temporal. We incor…
Riverine Land Cover Mapping through Semantic Segmentation of Multispectral Point Clouds
Department of Remote Sensing and Photogrammetry, Finnish Geospatial Research Institute FGI（遥感与摄影测量系，芬兰地理研究所）；Faculty of Technology, University of Turku（技术学院，图尔库大学）；Department of Geography and Geology, University of Turku（地理与地质系，图尔库大学）；Faculty of Geological Sciences, Complutense University of Madrid（地质科学学院，马德里卡洛斯三世大学）
Semantic segmentation of land cover classes is fundamental for agricultural and economic development work, from sustainable forestry to urban planning, yet existing training datasets have significant limitations. To generate an open and comprehensive training library of high resolution Earth imagery and high quality land cover classifications, public Sentinel-2 data at 10 m spatial resolution was matched with accurate GlobeLand30 labels from 2010, which were filtered by agreement with an intermediary Sentinel-2 classification at 20 m produced during atmospheric correction. Scene-level classifi…
In this paper we present our work on developing an automated system for land cover classification. This system takes a multiband satellite image of an area as input and outputs the land cover map of the area at the same resolution as the input. For this purpose convolutional machine learning models were trained in the task of predicting the land cover semantic segmentation of satellite images. This is a case of supervised learning. The land cover label data were taken from the CORINE Land Cover inventory and the satellite images were taken from the Copernicus hub. As for the model, U-Net archi…
Yale University（耶鲁大学）；Microsoft AI for Good（微软AI造福全球）；Microsoft Research（微软研究院）
We present simple algorithms for land cover change detection in the 2021 IEEE GRSS Data Fusion Contest. The task of the contest is to create high-resolution (1m / pixel) land cover change maps of a study area in Maryland, USA, given multi-resolution imagery and label data. We study several baseline models for this task and discuss directions for further research. See https://dfc2021.blob.core.windows.net/competition-data/dfc2021_index.txt for the data and https://github.com/calebrob6/dfc2021-msd-baseline for an implementation of these baselines.
In training machine learning models for land cover semantic segmentation there is a stark contrast between the availability of satellite imagery to be used as inputs and ground truth data to enable supervised learning. While thousands of new satellite images become freely available on a daily basis, getting ground truth data is still very challenging, time consuming and costly. In this paper we present Embedding Earth a self-supervised contrastive pre-training method for leveraging the large availability of satellite imagery to improve performance on downstream dense land cover classification …
Evaluation of Deep Learning Semantic Segmentation for Land Cover Mapping on Multispectral, Hyperspectral and High Spatial Aerial Imagery
Center for Space and Remote Sensing Research, National Central University（国立中央大学太空及遥测研究中心）；Department of Computer Science and Information Engineering, National Central University（国立中央大学资讯工程学系）
FLAIR: a Country-Scale Land Cover Semantic Segmentation Dataset From Multi-Source Optical Imagery
French National Institute of Geographical and Forest Information (IGN)（法国国家地理与森林信息研究所（IGN））；Univ Gustave Eiffel（古斯塔夫·埃菲尔大学）；National School of Geographic Sciences (ENSG)（国家地理科学学院（ENSG））；Ecole des Ponts（巴黎桥路学院）；CNRS（法国国家科学研究中心）
ICEYE Oy（ICEYE公司）；VTT Technical Research Centre of Finland（芬兰VTT技术研究中心）；Aalto University（阿尔托大学）
Land cover mapping is essential to monitoring the environment and understanding the effects of human activities on it. The automatic approaches to land cover mapping (i.e., image segmentation) mostly used traditional machine learning that requires heuristic feature design. On natural images, deep learning has outperformed traditional machine learning approaches for image segmentation. On remote sensing images, recent studies demonstrate successful applications of specific deep learning models to small-scale land cover mapping tasks (e.g., to classify wetland complexes). However, it is not read…
Landcover classification and change detection using remote sensing and machine learning: a case study of Western Fiji
Transitional Artificial Intelligence Research Group, School of Mathematics and Statistics, UNSW Sydney（过渡人工智能研究组，数学与统计学学院，UNSW悉尼）；School of Geosciences, University of Sydney（地质科学学院，悉尼大学）
Centre for Research-based Innovation Visual Intelligence（可视化研究型创新中心）
Multi-modality data is becoming readily available in remote sensing (RS) and can provide complementary information about the Earth's surface. Effective fusion of multi-modal information is thus important for various applications in RS, but also very challenging due to large domain differences, noise, and redundancies. There is a lack of effective and scalable fusion techniques for bridging multiple modality encoders and fully exploiting complementary information. To this end, we propose a new multi-modality network (MultiModNet) for land cover mapping of multi-modal remote sensing data based o…
Spatial Gated Multi-Layer Perceptron for Land Use and Land Cover Mapping
Simon Fraser University（西蒙菲莎大学）；Alipurduar Government Engineering and Management College（阿里布尔杜阿尔政府工程与管理学院）；Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）；Lancaster University（兰卡斯特大学）；Helmholtz-Zentrum Dresden-Rossendorf (HZDR)（德累斯顿-罗森多夫亥姆霍兹中心）；Institute of Advanced Research in Artificial Intelligence (IARAI)（人工智能高级研究所）
Freie Universität Berlin（柏林自由大学）；Rheinische Friedrich-Wilhelms-Universität Bonn（波恩莱茵弗里德里希·威廉大学）；Humboldt-Universität zu Berlin（柏林洪堡大学）
Remote sensing satellite data offer the unique possibility to map land use land cover transformations by providing spatially explicit information. However, detection of short-term processes and land use patterns of high spatial-temporal variability is a challenging task. We present a novel framework using multi-temporal TerraSAR-X data and machine learning techniques, namely Discriminative Markov Random Fields with spatio-temporal priors, and Import Vector Machines, in order to advance the mapping of land cover characterized by short-term changes. Our study region covers a current deforestatio…
INRAE（法国国家农业、食品与环境研究院）；University of Montpellier（蒙彼利埃大学）；CIRAD（法国国际农业发展研究中心）
Nowadays, there is a general agreement on the need to better characterize agricultural monitoring systems in response to the global changes. Timely and accurate land use/land cover mapping can support this vision by providing useful information at fine scale. Here, a deep learning approach is proposed to deal with multi-source land cover mapping at object level. The approach is based on an extension of Recurrent Neural Network enriched via an attention mechanism dedicated to multi-temporal data context. Moreover, a new hierarchical pretraining strategy designed to exploit specific domain knowl…
Wuhan University（武汉大学）；Chinese Academy of Sciences（中国科学院）；China Land Survey and Planning Institute（中国国土勘测规划院）
In recent years, large amount of high spatial-resolution remote sensing (HRRS) images are available for land-cover mapping. However, due to the complex information brought by the increased spatial resolution and the data disturbances caused by different conditions of image acquisition, it is often difficult to find an efficient method for achieving accurate land-cover classification with high-resolution and heterogeneous remote sensing images. In this paper, we propose a scheme to apply deep model obtained from labeled land-cover dataset to classify unlabeled HRRS images. The main idea is to r…
IRSTEA（法国国家农业、食品与环境研究院）；University of Montpellier（蒙彼利埃大学）
Radar and Optical Satellite Image Time Series (SITS) are sources of information that are commonly employed to monitor earth surfaces for tasks related to ecology, agriculture, mobility, land management planning and land cover monitoring. Many studies have been conducted using one of the two sources, but how to smartly combine the complementary information provided by radar and optical SITS is still an open challenge. In this context, we propose a new neural architecture for the combination of Sentinel-1 (S1) and Sentinel-2 (S2) imagery at object level, applied to a real-world land cover classi…
Land use and land cover mapping are essential to various fields of study, including forestry, agriculture, and urban management. Using earth observation satellites both facilitate and accelerate the task. Lately, deep learning methods have proven to be excellent at automating the mapping via semantic image segmentation. However, because deep neural networks require large amounts of labeled data, it is not easy to exploit the full potential of satellite imagery. Additionally, the land cover tends to differ in appearance from one region to another; therefore, having labeled data from one locatio…
Land-cover classification using remote sensing imagery is an important Earth observation task. Recently, land cover classification has benefited from the development of fully connected neural networks for semantic segmentation. The benchmark datasets available for training deep segmentation models in remote sensing imagery tend to be small, however, often consisting of only a handful of images from a single location with a single scale. This limits the models' ability to generalize to other datasets. Domain adaptation has been proposed to improve the models' generalization but we find these ap…
Regularly updated and accurate land cover maps are essential for monitoring 14 of the 17 Sustainable Development Goals. Multispectral satellite imagery provide high-quality and valuable information at global scale that can be used to develop land cover classification models. However, such a global application requires a geographically diverse training dataset. Here, we present LandCoverNet, a global training dataset for land cover classification based on Sentinel-2 observations at 10m spatial resolution. Land cover class labels are defined based on annual time-series of Sentinel-2, and verifie…
Accubits Technologies Inc（Accubits科技公司）；Acharya Narendra Dev College, University of Delhi（德里大学阿查里雅·纳伦德拉·德夫学院）；University of Calgary（卡尔加里大学）
Efficiently implementing remote sensing image classification with high spatial resolution imagery can provide a significant value in Land Use and Land Cover (LULC) classification. The new advances in remote sensing and deep learning technologies have facilitated the extraction of spatiotemporal information for LULC classification. Moreover, the diverse disciplines of science, including remote sensing, have utilised tremendous improvements in image classification by Convolutional Neural Networks (CNNs) with transfer learning. In this study, instead of training CNNs from scratch, the transfer le…
DI3CL: Contrastive Learning With Dynamic Instances and Contour Consistency for SAR Land-Cover Classification Foundation Model
Key Laboratory of Intelligent Perception and Image Understanding of Ministry of Education（教育部智能感知与图像理解重点实验室）；International Research Center for Intelligent Perception and Computation（智能感知与计算国际研究中心）；Joint International Research Laboratory of Intelligent Perception and Computation（智能感知与计算联合国际实验室）；School of Artificial Intelligence, Xidian University（西安电子科技大学人工智能学院）；Hangzhou Institute of Technology（杭州科技学院）
Sikkim Manipal Institute of Technology（锡金马尼帕尔理工学院）
Area of classifying satellite imagery has become a challenging task in current era where there is tremendous growth in settlement i.e. construction of buildings, roads, bridges, dam etc. This paper suggests an improvised k-means and Artificial Neural Network (ANN) classifier for land-cover mapping of Eastern Himalayan state Sikkim. The improvised k-means algorithm shows satisfactory results compared to existing methods that includes k-Nearest Neighbor and maximum likelihood classifier. The strength of the Artificial Neural Network (ANN) classifier lies in the fact that they are fast and have g…
The University of Tennessee, Knoxville（田纳西大学诺克斯维尔分校）；Applied Research Associates（应用研究联合公司）
Land cover classification of satellite imagery is an important step toward analyzing the Earth's surface. Existing models assume a closed-set setting where both the training and testing classes belong to the same label set. However, due to the unique characteristics of satellite imagery with an extremely vast area of versatile cover materials, the training data are bound to be non-representative. In this paper, we study the problem of open-set land cover classification that identifies the samples belonging to unknown classes during testing, while maintaining performance on known classes. Altho…
Spatial-Frequency Gated Swin Transformer for Cross-Sensor Remote Sensing Super-Resolution
Space Applications Centre, ISRO, Ahmedabad, India（印度航天研究组织阿赫迈德亚布德研究中心）；Centre of Studies in Resources Engineering, Indian Institute of Technology Bombay, India（印度理工学院孟买资源工程研究学院）；New L J Institute of Engineering and Technology, Ahmedabad, India（阿赫迈德亚布德新LJ工程与技术学院）；Pandit Deendayal Energy University, Gandhinagar, India（潘迪特·德恩达尔能源大学）；GLS University, Ahmedabad, India（阿赫迈德亚布德GLS大学）
RIKEN Center for Advanced Intelligence Project（理化学研究所先进智能项目中心）；Inria（法国国家信息与自动化研究所）；Munich University of Applied Sciences（慕尼黑应用科学大学）；the University of Tokyo（东京大学）
We synthesize both optical RGB and synthetic aperture radar (SAR) remote sensing images from land cover maps and auxiliary raster data using generative adversarial networks (GANs). In remote sensing, many types of data, such as digital elevation models (DEMs) or precipitation maps, are often not reflected in land cover maps but still influence image content or structure. Including such data in the synthesis process increases the quality of the generated images and exerts more control on their characteristics. Spatially adaptive normalization layers fuse both inputs and are applied to a full-bl…
LandSegmenter: Towards a Flexible Foundation Model for Land Use and Land Cover Mapping
Chair of Data Science in Earth Observation, Technical University of Munich（地球观测数据科学教授职位，慕尼黑技术大学）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心（MCML））
Sustainability of the global environment is dependent on the accurate land cover information over large areas. Even with the increased number of satellite systems and sensors acquiring data with improved spectral, spatial, radiometric and temporal characteristics and the new data distribution policy, most existing land cover datasets were derived from a pixel-based single-date multi-spectral remotely sensed image with low accuracy. To improve the accuracy, the bottleneck is how to develop an accurate and effective image classification technique. By incorporating and utilizing the complete mult…
This work presents SeasoNet, a new large-scale multi-label land cover and land use scene understanding dataset. It includes $1\,759\,830$ images from Sentinel-2 tiles, with 12 spectral bands and patch sizes of up to $ 120 \ \mathrm{px} \times 120 \ \mathrm{px}$. Each image is annotated with large scale pixel level labels from the German land cover model LBM-DE2018 with land cover classes based on the CORINE Land Cover database (CLC) 2018 and a five times smaller minimum mapping unit (MMU) than the original CLC maps. We provide pixel synchronous examples from all four seasons, plus an additiona…
Nowadays, modern Earth Observation systems continuously collect massive amounts of satellite information. The unprecedented possibility to acquire high resolution Satellite Image Time Series (SITS) data (series of images with high revisit time period on the same geographical area) is opening new opportunities to monitor the different aspects of the Earth Surface but, at the same time, it is raising up new challenges in term of suitable methods to analyze and exploit such huge amount of rich and complex image data. One of the main task associated to SITS data analysis is related to land cover m…
University of Minnesota（明尼苏达大学）；University of Pittsburgh（匹兹堡大学）
Land cover mapping is essential for monitoring global environmental change and managing natural resources. Unfortunately, traditional classification models are plagued by limited training data available in existing land cover products and data heterogeneity over space and time. In this survey, we provide a structured and comprehensive overview of challenges in land cover mapping and machine learning methods used to address these problems. We also discuss the gaps and opportunities that exist for advancing research in this promising direction.
Land use as contained in geospatial databases constitutes an essential input for different applica-tions such as urban management, regional planning and environmental monitoring. In this paper, a hierarchical deep learning framework is proposed to verify the land use information. For this purpose, a two-step strategy is applied. First, given high-resolution aerial images, the land cover information is determined. To achieve this, an encoder-decoder based convolutional neural net-work (CNN) is proposed. Second, the pixel-wise land cover information along with the aerial images serves as input f…
TU Kaiserslautern（凯撒斯劳滕工业大学）；German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心（DFKI））
In this paper, we address the challenge of land use and land cover classification using Sentinel-2 satellite images. The Sentinel-2 satellite images are openly and freely accessible provided in the Earth observation program Copernicus. We present a novel dataset based on Sentinel-2 satellite images covering 13 spectral bands and consisting out of 10 classes with in total 27,000 labeled and geo-referenced images. We provide benchmarks for this novel dataset with its spectral bands using state-of-the-art deep Convolutional Neural Network (CNNs). With the proposed novel dataset, we achieved an ov…
RIKEN AIP（理化学研究所先进智能项目中心）；The University of Tokyo（东京大学）
We introduce OpenEarthMap, a benchmark dataset, for global high-resolution land cover mapping. OpenEarthMap consists of 2.2 million segments of 5000 aerial and satellite images covering 97 regions from 44 countries across 6 continents, with manually annotated 8-class land cover labels at a 0.25--0.5m ground sampling distance. Semantic segmentation models trained on the OpenEarthMap generalize worldwide and can be used as off-the-shelf models in a variety of applications. We evaluate the performance of state-of-the-art methods for unsupervised domain adaptation and present challenging problem s…
Time series data of urban land cover is of great utility in analyzing urban growth patterns, changes in distribution of impervious surface and vegetation and resulting impacts on urban micro climate. While Landsat data is ideal for such analysis due to the long time series of free imagery, traditional per-pixel hard classification fails to yield full potential of the Landsat data. This paper proposes a sub-pixel classification method that leverages the temporal overlap of Landsat-5 TM and Resourcesat-1 LISS-IV sensors. We train a convolutional neural network to predict fractional land cover ma…
University of Minnesota（明尼苏达大学）；University of Pittsburgh（匹兹堡大学）
The availability of massive earth observing satellite data provide huge opportunities for land use and land cover mapping. However, such mapping effort is challenging due to the existence of various land cover classes, noisy data, and the lack of proper labels. Also, each land cover class typically has its own unique temporal pattern and can be identified only during certain periods. In this article, we introduce a novel architecture that incorporates the UNet structure with Bidirectional LSTM and Attention mechanism to jointly exploit the spatial and temporal nature of satellite data and to b…
Norwegian Institute for Nature Research - NINA（挪威自然研究所）
Widely used European land cover maps such as CORINE are produced at medium spatial resolutions (100 m) and rely on diverse data with complex workflows requiring significant institutional capacity. We present a high resolution (10 m) land cover map (ELC10) of Europe based on a satellite-driven machine learning workflow that is annually updatable. A Random Forest classification model was trained on 70K ground-truth points from the LUCAS (Land Use/Cover Area frame Survey) dataset. Within the Google Earth Engine cloud computing environment, the ELC10 map can be generated from approx. 700 TB of Sen…
