Improvement in Land Cover and Crop Classification based on Temporal Features Learning from Sentinel-2 Data Using Recurrent-Convolutional Neural Network (R-CNN)
The increasing spatial and temporal resolution of globally available satellite images, such as provided by Sentinel-2, creates new possibilities for researchers to use freely available multi-spectral optical images, with decametric spatial resolution and more frequent revisits for remote sensing applications such as land cover and crop classification (LC&CC), agricultural monitoring and management, environment monitoring. Existing solutions dedicated to cropland mapping can be categorized based on per-pixel based and object-based. However, it is still challenging when more classes of agricultural crops are considered at a massive scale. In this paper, a novel and optimal deep learning model for pixel-based LC&CC is developed and implemented based on Recurrent Neural Networks (RNN) in combination with Convolutional Neural Networks (CNN) using multi-temporal sentinel-2 imagery of central north part of Italy, which has diverse agricultural system dominated by economic crop types. The proposed methodology is capable of automated feature extraction by learning time correlation of multiple images, which reduces manual feature engineering and modeling crop phenological stages. Fifteen classes, including major agricultural crops, were considered in this study. We also tested other widely used traditional machine learning algorithms for comparison such as support vector machine SVM, random forest (RF), Kernal SVM, and gradient boosting machine, also called XGBoost. The overall accuracy achieved by our proposed Pixel R-CNN was 96.5%, which showed considerable improvements in comparison with existing mainstream methods. This study showed that Pixel R-CNN based model offers a highly accurate way to assess and employ time-series data for multi-temporal classification tasks.
Goldman School of Public Policy, University of California, Berkeley（加州大学伯克利分校高盛公共政策学院）
High resolution crop type maps are an important tool for improving food security, and remote sensing is increasingly used to create such maps in regions that possess ground truth labels for model training. However, these labels are absent in many regions, and models trained in other regions on typical satellite features, such as those from optical sensors, often exhibit low performance when transferred. Here we explore the use of NASA's Global Ecosystem Dynamics Investigation (GEDI) spaceborne lidar instrument, combined with Sentinel-2 optical data, for crop type mapping. Using data from three…
Fine-grained Hierarchical Crop Type Classification from Integrated Hyperspectral EnMAP Data and Multispectral Sentinel-2 Time Series: A Large-scale Dataset and Dual-stream Transformer Method
Jockey Club STEM Lab of Quantitative Remote Sensing, Department of Geography, The University of Hong Kong, Hong Kong, China（香港大学地理系定量遥感实验室）；Department of Aerospace Intelligent Science and Technology, School of Astronautics, Beihang University, Beijing, China（北航航天智能科学与技术系）
Crop type maps are critical for tracking agricultural land use and estimating crop production. Remote sensing has proven an efficient and reliable tool for creating these maps in regions with abundant ground labels for model training, yet these labels remain difficult to obtain in many regions and years. NASA's Global Ecosystem Dynamics Investigation (GEDI) spaceborne lidar instrument, originally designed for forest monitoring, has shown promise for distinguishing tall and short crops. In the current study, we leverage GEDI to develop wall-to-wall maps of short vs tall crops on a global scale …
GFZ German Research Centre for Geosciences（德国地学研究中心）；Universitat de València（瓦伦西亚大学）；University of Twente（特文特大学）；Nanjing University（南京大学）
Satellite remote sensing has been widely used in the last decades for agricultural applications, {both for assessing vegetation condition and for subsequent yield prediction.} Existing remote sensing-based methods to estimate gross primary productivity (GPP), which is an important variable to indicate crop photosynthetic function and stress, typically rely on empirical or semi-empirical approaches, which tend to over-simplify photosynthetic mechanisms. In this work, we take advantage of all parallel developments in mechanistic photosynthesis modeling and satellite data availability for advance…
An accurate and precise understanding of global irrigation usage is crucial for a variety of climate science efforts. Irrigation is highly energy-intensive, and as population growth continues at its current pace, increases in crop need and water usage will have an impact on climate change. Precise irrigation data can help with monitoring water usage and optimizing agricultural yield, particularly in developing countries. Irrigation data, in tandem with precipitation data, can be used to predict water budgets as well as climate and weather modeling. With our research, we produce an irrigation p…
Otto-von-Guericke University Magdeburg（马格德堡奥托·冯·格里克大学）；BASF Digital Farming GmbH（巴斯夫数字农业有限公司）
Landsat-8 (NASA) and Sentinel-2 (ESA) are two prominent multi-spectral imaging satellite projects that provide publicly available data. The multi-spectral imaging sensors of the satellites capture images of the earth's surface in the visible and infrared region of the electromagnetic spectrum. Since the majority of the earth's surface is constantly covered with clouds, which are not transparent at these wavelengths, many images do not provide much information. To increase the temporal availability of cloud-free images of a certain area, one can combine the observations from multiple sources. H…
IRSTEA（法国国家农业、食品与环境研究院）；University of Montpellier（蒙彼利埃大学）
Radar and Optical Satellite Image Time Series (SITS) are sources of information that are commonly employed to monitor earth surfaces for tasks related to ecology, agriculture, mobility, land management planning and land cover monitoring. Many studies have been conducted using one of the two sources, but how to smartly combine the complementary information provided by radar and optical SITS is still an open challenge. In this context, we propose a new neural architecture for the combination of Sentinel-1 (S1) and Sentinel-2 (S2) imagery at object level, applied to a real-world land cover classi…
In this article, we investigate several structured deep learning models for crop type classification on multi-spectral time series. In particular, our aim is to assess the respective importance of spatial and temporal structures in such data. With this objective, we consider several designs of convolutional, recurrent, and hybrid neural networks, and assess their performance on a large dataset of freely available Sentinel-2 imagery. We find that the best-performing approaches are hybrid configurations for which most of the parameters (up to 90%) are allocated to modeling the temporal structure…
Silesian University of Technology（西里西亚工业大学）；KP Labs（KP实验室）；European Space Agency（欧洲空间局）
Maintaining farm sustainability through optimizing the agricultural management practices helps build more planet-friendly environment. The emerging satellite missions can acquire multi- and hyperspectral imagery which captures more detailed spectral information concerning the scanned area, hence allows us to benefit from subtle spectral features during the analysis process in agricultural applications. We introduce an approach for extracting 2.5 m cultivated land maps from 10 m Sentinel-2 multispectral image series which benefits from a compact graph convolutional neural network. The experimen…
ICEYE Oy（ICEYE公司）；VTT Technical Research Centre of Finland（芬兰VTT技术研究中心）；Aalto University（阿尔托大学）
Land cover mapping is essential to monitoring the environment and understanding the effects of human activities on it. The automatic approaches to land cover mapping (i.e., image segmentation) mostly used traditional machine learning that requires heuristic feature design. On natural images, deep learning has outperformed traditional machine learning approaches for image segmentation. On remote sensing images, recent studies demonstrate successful applications of specific deep learning models to small-scale land cover mapping tasks (e.g., to classify wetland complexes). However, it is not read…
University of Winnipeg（温尼伯大学）；Norwegian University of Science & Technology (NTNU) Gjøvik（挪威科技大学（NTNU）约维克校区）
The availability of frequent and cost-free satellite images is in growing demand in the research world. Such satellite constellations as Landsat 8 and Sentinel-2 provide a massive amount of valuable data daily. However, the discrepancy in the sensors' characteristics of these satellites makes it senseless to use a segmentation model trained on either dataset and applied to another, which is why domain adaptation techniques have recently become an active research area in remote sensing. In this paper, an experiment of domain adaptation through style-transferring is conducted using the HRSemI2I …
Multitask GLocal OBIA-Mamba for Sentinel-2 Landcover Mapping
Department of Geomatics Engineering, University of Calgary（测绘工程系，卡尔加里大学）；Department of Geography, Faculty of Arts, The University of Jordan（地理系，艺术学院，约旦大学）
Land use classification of low resolution spatial imagery is one of the most extensively researched fields in remote sensing. Despite significant advancements in satellite technology, high resolution imagery lacks global coverage and can be prohibitively expensive to procure for extended time periods. Accurately classifying land use change without high resolution imagery offers the potential to monitor vital aspects of global development agenda including climate smart agriculture, drought resistant crops, and sustainable land management. Utilizing a combination of capsule layers and long-short…
High-resolution canopy height map in the Landes forest (France) based on GEDI, Sentinel-1, and Sentinel-2 data with a deep learning approach
Laboratoire des Sciences du Climat et de l'Environnement, LSCE/IPSL, CEA-CNRS-SCE32UVSQ, Université Paris Saclay（巴黎萨克雷大学气候与环境科学实验室）；Kayrros SAS（Kayrros SAS公司）；IGN, Laboratoire d’Inventaire Forestier（法国国家地理与森林信息研究所森林清查实验室）；INRAE, UMR TETIS, Université de Montpellier, AgroParisTech（法国国家农业食品与环境研究院、蒙彼利埃大学、巴黎高科农业学院）；Department of Geosciences and Natural Resource Management, University of Copenhagen（哥本哈根大学地球科学与自然资源管理系）；IGN, Service de l'Information Statistique, Forestière et Environnementale（法国国家地理与森林信息研究所统计、森林与环境信息服务处）；CESBIO, Université de Toulouse, CNES/CNRS/INRAE/IRD/UPS（图卢兹大学生物圈空间研究中心）；ISPA, UMR 1391, INRA Nouvelle-Aquitaine（法国国家农业研究院新阿基坦大区农业与环境相互作用研究所）
University of Sannio（萨尼奥大学）；Massachusetts Institute of Technology (MIT)（麻省理工学院（MIT））
In this paper, the authors aim to combine the latest state of the art models in image recognition with the best publicly available satellite images to create a system for landslide risk mitigation. We focus first on landslide detection and further propose a similar system to be used for prediction. Such models are valuable as they could easily be scaled up to provide data for hazard evaluation, as satellite imagery becomes increasingly available. The goal is to use satellite images and correlated data to enrich the public repository of data and guide disaster relief efforts for locating precis…
TU Kaiserslautern（凯撒斯劳滕工业大学）；German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心（DFKI））
In this paper, we address the challenge of land use and land cover classification using Sentinel-2 satellite images. The Sentinel-2 satellite images are openly and freely accessible provided in the Earth observation program Copernicus. We present a novel dataset based on Sentinel-2 satellite images covering 13 spectral bands and consisting out of 10 classes with in total 27,000 labeled and geo-referenced images. We provide benchmarks for this novel dataset with its spectral bands using state-of-the-art deep Convolutional Neural Network (CNNs). With the proposed novel dataset, we achieved an ov…
Regularly updated and accurate land cover maps are essential for monitoring 14 of the 17 Sustainable Development Goals. Multispectral satellite imagery provide high-quality and valuable information at global scale that can be used to develop land cover classification models. However, such a global application requires a geographically diverse training dataset. Here, we present LandCoverNet, a global training dataset for land cover classification based on Sentinel-2 observations at 10m spatial resolution. Land cover class labels are defined based on annual time-series of Sentinel-2, and verifie…
Norwegian Institute for Nature Research - NINA（挪威自然研究所）
Widely used European land cover maps such as CORINE are produced at medium spatial resolutions (100 m) and rely on diverse data with complex workflows requiring significant institutional capacity. We present a high resolution (10 m) land cover map (ELC10) of Europe based on a satellite-driven machine learning workflow that is annually updatable. A Random Forest classification model was trained on 70K ground-truth points from the LUCAS (Land Use/Cover Area frame Survey) dataset. Within the Google Earth Engine cloud computing environment, the ELC10 map can be generated from approx. 700 TB of Sen…
Technical University of Munich（慕尼黑工业大学）；Univ. Bretagne Sud（南布列塔尼大学）；IRISA（法国国家信息与自动化研究所/雷恩信息与系统研究所（注：IRISA常见中文名为雷恩信息与自动化研究所，为法国国家科研中心与多校共建的独立研究机构，此处保留原名并给出常用译名））
We present Breizhcrops, a novel benchmark dataset for the supervised classification of field crops from satellite time series. We aggregated label data and Sentinel-2 top-of-atmosphere as well as bottom-of-atmosphere time series in the region of Brittany (Breizh in local language), north-east France. We compare seven recently proposed deep neural networks along with a Random Forest baseline. The dataset, model (re-)implementations and pre-trained model weights are available at the associated GitHub repository (https://github.com/dl4sits/BreizhCrops) that has been designed with applicability fo…
Scattered trees outside of dense, closed-canopy forests are very important for carbon sequestration, supporting livelihoods, maintaining ecosystem integrity, and climate change adaptation and mitigation. In contrast to trees inside of closed-canopy forests, not much is known about the spatial extent and distribution of scattered trees at a global scale. Due to the cost of high-resolution satellite imagery, global monitoring systems rely on medium-resolution satellites to monitor land use. Here we present a globally consistent method to identify trees with canopy diameters greater than three me…
Smith School of Enterprise and the Environment（史密斯企业与环境学院）；School of Geography and the Environment（地理与环境学院）；University of Oxford（牛津大学）
Earth observation offers new insight into anthropogenic changes to nature, and how these changes are effecting (and are effected by) the built environment and the real economy. With the global availability of medium-resolution (10-30m) synthetic aperture radar (SAR) Sentinel-1 and multispectral Sentinel-2 imagery, machine learning can be employed to offer these insights at scale, unbiased to the reporting of companies and countries. In this paper, I introduce DeepSentinel, a data pipeline and experimentation framework for producing general-purpose semantic embeddings of paired Sentinel-1 and S…
Predicting Crop Yield With Machine Learning: An Extensive Analysis Of Input Modalities And Models On a Field and sub-field Level
University of Kaiserslautern-Landau (RPTU)（凯泽斯劳滕-兰道大学）；German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心）；Vision Impulse GmbH（Vision Impulse 有限责任公司）
Improved flood mapping for efficient policy design by fusion of Sentinel-1, Sentinel-2, and Landsat-9 imagery to identify population and infrastructure exposed to floods
Lahore University of Management Sciences (LUMS)（拉合尔管理科学大学（LUMS））
Deep Neural Networks (DNNs) are getting increasing attention to deal with Land Cover Classification (LCC) relying on Satellite Image Time Series (SITS). Though high performances can be achieved, the rationale of a prediction yielded by a DNN often remains unclear. An architecture expressing predictions with respect to input channels is thus proposed in this paper. It relies on convolutional layers and an attention mechanism weighting the importance of each channel in the final classification decision. The correlation between channels is taken into account to set up shared kernels and lower mod…
Towards more efficient agricultural practices via transformer-based crop type classification
Gobierno del estado de Jalisco（哈利斯科州政府）；TecMM Unidad Zapopan（TecMM萨波潘分部）；Climate Change AI（气候变化人工智能）；Kabarak University（卡巴拉大学）；Aspia Space（阿斯皮亚航天）；Columbia University（哥伦比亚大学）
SenCLIP: Enhancing zero-shot land-use mapping for Sentinel-2 with ground-level prompting
Mediterranean Agronomic Institute of Montpellier - CIHEAM-IAMM（地中海农业研究院-CIHEAM-IAMM）；Inria（法国国家信息与自动化技术研究院）；INRAE（法国国家农业研究咨询中心）；Cirad（国际热带农业研究中心）；UMR TETIS（TETIS联合研究单位）；Univ. of Montpellier（蒙彼利埃大学）
HLS-GPT: A Generative Pretrained Transformer (GPT) for Continental-Scale NASA Harmonized Landsat and Sentinel-2 (HLS) Reflectance Reconstruction Across All Bands on Arbitrary Dates
Geospatial Sciences Center of Excellence, Department of Geography and Geospatial Sciences, South Dakota State University（地理科学卓越中心，地理与空间科学系，南达科他州立大学）；Department of Geography, Environment, and Spatial Sciences, & Center for Global Change and Earth Observations, Michigan State University（地理、环境与空间科学系，以及全球变化与地球观测中心，密歇根州立大学）
Deep Learning for automated multi-scale functional field boundaries extraction using multi-date Sentinel-2 and PlanetScope imagery: Case Study of Netherlands and Pakistan
A multi-temporal multi-spectral attention-augmented deep convolution neural network with contrastive learning for crop yield prediction
Department of Computer Science and Engineering, Indian Institute of Technology (IIT) Indore, Indore 453552, India（计算机科学与工程系，印度理工学院（IIT）印多尔，印多尔453552，印度）
Evaluating the Role of Training Data Origin for Country-Scale Cropland Mapping in Data-Scarce Regions: A Case Study of Nigeria
ETH Zurich（苏黎世联邦理工学院）；Swiss Federal Laboratories for Material Science and Technology（瑞士材料科学与技术联邦实验室）；Swiss Data Science Center（瑞士数据科学中心）；EPFL（洛桑联邦理工学院）；Wageningen University & Research（瓦赫宁根大学与研究中心）
