Optical satellite sensors cannot see the Earth's surface through clouds. Despite the periodic revisit cycle, image sequences acquired by Earth observation satellites are therefore irregularly sampled in time. State-of-the-art methods for crop classification (and other time series analysis tasks) rely on techniques that implicitly assume regular temporal spacing between observations, such as recurrent neural networks (RNNs). We propose to use neural ordinary differential equations (NODEs) in combination with RNNs to classify crop types in irregularly spaced image sequences. The resulting ODE-RNN models consist of two steps: an update step, where a recurrent unit assimilates new input data into the model's hidden state; and a prediction step, in which NODE propagates the hidden state until the next observation arrives. The prediction step is based on a continuous representation of the latent dynamics, which has several advantages. At the conceptual level, it is a more natural way to describe the mechanisms that govern the phenological cycle. From a practical point of view, it makes it possible to sample the system state at arbitrary points in time, such that one can integrate observations whenever they are available, and extrapolate beyond the last observation. Our experiments show that ODE-RNN indeed improves classification accuracy over common baselines such as LSTM, GRU, and temporal convolution. The gains are most prominent in the challenging scenario where only few observations are available (i.e., frequent cloud cover). Moreover, we show that the ability to extrapolate translates to better classification performance early in the season, which is important for forecasting.
Improving Data Efficiency for Plant Cover Prediction with Label Interpolation and Monte-Carlo Cropping
Friedrich Schiller University（弗里德里希·席勒大学）；German Centre for Integrative Biodiversity Research (iDiv) Halle-Jena-Leipzig（德国综合生物多样性研究中心（iDiv）哈雷-耶拿-莱比锡）；Michael Stifel Center Jena（耶拿米夏埃尔·施蒂费尔中心）
Applying agrochemicals is the default procedure for conventional weed control in crop production, but has negative impacts on the environment. Robots have the potential to treat every plant in the field individually and thus can reduce the required use of such chemicals. To achieve that, robots need the ability to identify crops and weeds in the field and must additionally select effective treatments. While certain types of weed can be treated mechanically, other types need to be treated by (selective) spraying. In this paper, we present an approach that provides the necessary information for …
The increasing spatial and temporal resolution of globally available satellite images, such as provided by Sentinel-2, creates new possibilities for researchers to use freely available multi-spectral optical images, with decametric spatial resolution and more frequent revisits for remote sensing applications such as land cover and crop classification (LC&CC), agricultural monitoring and management, environment monitoring. Existing solutions dedicated to cropland mapping can be categorized based on per-pixel based and object-based. However, it is still challenging when more classes of agricultu…
California Crop Yield Benchmark: Combining Satellite Image, Climate, Evapotranspiration, and Soil Data Layers for County-Level Yield Forecasting of Over 70 Crops
University of California, Davis（加州大学戴维斯分校）；Texas A&M University-Corpus Christi（德克萨斯A&M大学-科斯蒂西）；AI Institute for Food Systems（人工智能食品系统研究所）
Hubei University of Technology（湖北工业大学）；Wuhan University（武汉大学）
Land cover classification is a multi-class segmentation task to classify each pixel into a certain natural or man-made category of the earth surface, such as water, soil, natural vegetation, crops, and human infrastructure. Limited by hardware computational resources and memory capacity, most existing studies preprocessed original remote sensing images by down sampling or cropping them into small patches less than 512*512 pixels before sending them to a deep neural network. However, down sampling images incurs spatial detail loss, renders small segments hard to discriminate, and reverses the s…
ETH Zurich（苏黎世联邦理工学院）；Agroscope（瑞士农业研究中心）；Federal Office for Agriculture, Switzerland（瑞士联邦农业办公室）；University of Zurich（苏黎世大学）
The aim of this paper is to map agricultural crops by classifying satellite image time series. Domain experts in agriculture work with crop type labels that are organised in a hierarchical tree structure, where coarse classes (like orchards) are subdivided into finer ones (like apples, pears, vines, etc.). We develop a crop classification method that exploits this expert knowledge and significantly improves the mapping of rare crop types. The three-level label hierarchy is encoded in a convolutional, recurrent neural network (convRNN), such that for each pixel the model predicts three labels a…
Land use classification of low resolution spatial imagery is one of the most extensively researched fields in remote sensing. Despite significant advancements in satellite technology, high resolution imagery lacks global coverage and can be prohibitively expensive to procure for extended time periods. Accurately classifying land use change without high resolution imagery offers the potential to monitor vital aspects of global development agenda including climate smart agriculture, drought resistant crops, and sustainable land management. Utilizing a combination of capsule layers and long-short…
Optical and radar satellite time series are synergetic: optical images contain rich spectral information, while C-band radar captures useful geometrical information and is immune to cloud cover. Motivated by the recent success of temporal attention-based methods across multiple crop mapping tasks, we propose to investigate how these models can be adapted to operate on several modalities. We implement and evaluate multiple fusion schemes, including a novel approach and simple adjustments to the training procedure, significantly improving performance and efficiency with little added complexity. …
University of Minnesota Twin Cities（明尼苏达大学双城分校）；Ant Group（蚂蚁集团）；Texas Tech University（德克萨斯理工大学）；Institute of Geographic Sciences and Natural Resources Research, Chinese Academy of Sciences（中国科学院地理科学与资源研究所）；Stanford University（斯坦福大学）
Land cover classification in remote sensing is often faced with the challenge of limited ground truth. Incorporating historical information has the potential to significantly lower the expensive cost associated with collecting ground truth and, more importantly, enable early- and in-season mapping that is helpful to many pre-harvest decisions. In this study, we propose a new approach that can effectively transfer knowledge about the topology (i.e. relative position) of different crop types in the spectral feature space (e.g. the histogram of SWIR1 vs RDEG1 bands) to generate labels, thereby su…
MT-CYP-Net: Multi-Task Network for Pixel-Level Crop Yield Prediction Under Very Few Samples
State Key Laboratory of Water Resources Engineering and Management, Wuhan University（水利水电工程与管理国家重点实验室，武汉大学）；School of Computer Science, Wuhan University（武汉大学计算机学院）；State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University（测绘遥感信息工程国家重点实验室，武汉大学）；College of Agricultural Science and Engineering, Hohai University（河海大学农业科学与工程学院）
GAF AG（GAF股份公司）；Technical University of Munich (TUM)（慕尼黑工业大学）
We present EuroCrops, a dataset based on self-declared field annotations for training and evaluating methods for crop type classification and mapping, together with its process of acquisition and harmonisation. By this, we aim to enrich the research efforts and discussion for data-driven land cover classification via Earth observation and remote sensing. Additionally, through inclusion of self-declarations gathered in the scope of subsidy control from all countries of the European Union (EU), this dataset highlights the difficulties and pitfalls one comes across when operating on a transnation…
Indraprastha Institute of Information Technology, Delhi（德里英德拉普拉斯塔信息技术学院）；M S Swaminathan Research Foundation（M S 斯瓦米纳坦研究基金会）
The availability of well-curated datasets has driven the success of Machine Learning (ML) models. Despite the increased access to earth observation data for agriculture, there is a scarcity of curated, labelled datasets, which limits the potential of its use in training ML models for remote sensing (RS) in agriculture. To this end, we introduce a first-of-its-kind dataset, SICKLE, having time-series images at different spatial resolutions from 3 different satellites, annotated with multiple key cropping parameters for paddy cultivation for the Cauvery Delta region in Tamil Nadu, India. The dat…
Machine vision for precision agriculture has attracted considerable research interest in recent years. The goal of this paper is to develop an end-to-end cranberry health monitoring system to enable and support real time cranberry over-heating assessment to facilitate informed decisions that may sustain the economic viability of the farm. Toward this goal, we propose two main deep learning-based modules for: 1) cranberry fruit segmentation to delineate the exact fruit regions in the cranberry field image that are exposed to sun, 2) prediction of cloud coverage conditions and sun irradiance to …
Harbin Institute of Technology（哈尔滨工业大学）；SenseTime Group Limited（商汤科技有限公司）
Recently, FCNs based methods have made great progress in semantic segmentation. Different with ordinary scenes, satellite image owns specific characteristics, which elements always extend to large scope and no regular or clear boundaries. Therefore, effective mid-level structure information extremely missing, precise pixel-level classification becomes tough issues. In this paper, a Dense Fusion Classmate Network (DFCNet) is proposed to adopt in land cover classification.
From Seedling to Harvest: The GrowingSoy Dataset for Weed Detection in Soy Crops via Instance Segmentation
Universidade Federal de Santa Maria（圣玛丽亚联邦大学）；University of Tsukuba（筑波大学）；Universidad Tecnológica del Uruguay（乌拉圭科技大学）；Wageningen University & Research（瓦赫宁根大学与研究中心）
Earth observation (EO) sensors deliver data with daily or weekly temporal resolution. Most land use and land cover (LULC) approaches, however, expect cloud-free and mono-temporal observations. The increasing temporal capabilities of today's sensors enables the use of temporal, along with spectral and spatial features. Domains, such as speech recognition or neural machine translation, work with inherently temporal data and, today, achieve impressive results using sequential encoder-decoder structures. Inspired by these sequence-to-sequence models, we adapt an encoder structure with convolutiona…
Murdoch University（莫道克大学）；Centre for Crop and Food Innovation（作物与食品创新中心）；Food Futures Institute（食品未来研究所）；Department of Primary Industries and Regional Development, Western Australia（西澳大利亚州初级产业与区域发展部）；Centre for Sustainable Farming Systems（可持续农业系统中心）；Centre of Biosecurity and One Health, Harry Butler Institute（哈里·巴特勒研究所生物安全与同一健康中心）
The rapid advances in Deep Learning (DL) techniques have enabled rapid detection, localisation, and recognition of objects from images or videos. DL techniques are now being used in many applications related to agriculture and farming. Automatic detection and classification of weeds can play an important role in weed management and so contribute to higher yields. Weed detection in crops from imagery is inherently a challenging problem because both weeds and crops have similar colours ('green-on-green'), and their shapes and texture can be very similar at the growth phase. Also, a crop in one s…
Politecnico di Torino（都灵理工大学）；Politecnico Interdepartmental Centre for Service Robotics（都灵理工大学跨部门服务机器人中心）；Università degli Studi di Torino（都灵大学）；Institute of Electronics, Computer and Telecommunication Engineering of the National Research Council of Italy（意大利国家研究委员会电子、计算机与电信工程研究所）
Precision agriculture is considered to be a fundamental approach in pursuing a low-input, high-efficiency, and sustainable kind of agriculture when performing site-specific management practices. To achieve this objective, a reliable and updated description of the local status of crops is required. Remote sensing, and in particular satellite-based imagery, proved to be a valuable tool in crop mapping, monitoring, and diseases assessment. However, freely available satellite imagery with low or moderate resolutions showed some limits in specific agricultural applications, e.g., where crops are gr…
Data-driven Prediction of Species-Specific Plant Responses to Spectral-Shifting Films from Leaf Phenotypic and Photosynthetic Traits
Smart Farm Research Center, Korea Institute of Science and Technology（韩国科学技术院智能农场研究中心）；Research Institute of Agriculture and Life Sciences, Seoul National University（首尔国立大学农业与生命科学研究院）；Department of Agriculture, Forestry and Bioresources, Seoul National University（首尔国立大学农业、林业与生物资源系）
Semantic segmentation of land cover classes is fundamental for agricultural and economic development work, from sustainable forestry to urban planning, yet existing training datasets have significant limitations. To generate an open and comprehensive training library of high resolution Earth imagery and high quality land cover classifications, public Sentinel-2 data at 10 m spatial resolution was matched with accurate GlobeLand30 labels from 2010, which were filtered by agreement with an intermediary Sentinel-2 classification at 20 m produced during atmospheric correction. Scene-level classifi…
University of Minnesota（明尼苏达大学）；University of Pittsburgh（匹兹堡大学）
The availability of massive earth observing satellite data provide huge opportunities for land use and land cover mapping. However, such mapping effort is challenging due to the existence of various land cover classes, noisy data, and the lack of proper labels. Also, each land cover class typically has its own unique temporal pattern and can be identified only during certain periods. In this article, we introduce a novel architecture that incorporates the UNet structure with Bidirectional LSTM and Attention mechanism to jointly exploit the spatial and temporal nature of satellite data and to b…
FLNet: Flood-Induced Agriculture Damage Assessment using Super Resolution of Satellite Images
Annam.AI CoE, MoE, Indian Institute of Technology Ropar, Punjab, India（Annam.AI联合研究所、教育部、印度理工学院罗帕尔分校）；Department of Mathematical Sciences, Rajiv Gandhi Institute of Petroleum Technology, Jais, UP, India（数学科学系、拉贾吉安石油技术学院）；Department of Computer Science and Engineering, Indian Institute of Technology Ropar, Punjab, India（计算机科学与工程系、印度理工学院罗帕尔分校）
