Multi-task fully convolutional network for tree species mapping in dense forests using small training hyperspectral data
Pontifical Catholic University of Rio de Janeiro（里约热内卢天主教大学）；IBM Research（IBM研究院）；McMaster University（麦克马斯特大学）；National Institute for Space Research (INPE)（巴西国家空间研究所）；Santa Catarina State University (UDESC)（圣卡塔琳娜州立大学）
Laura Elena Cué La Rosa, Camile Sothe, Raul Queiroz Feitosa, Cláudia Maria de Almeida, Marcos Benedito Schimalski, Dario Augusto Borges Oliveira
备注：
Full version of preprint accepted at ISPRS Journal of Photogrammetry and Remote Sensing
This work proposes a multi-task fully convolutional architecture for tree species mapping in dense forests from sparse and scarce polygon-level annotations using hyperspectral UAV-borne data. Our model implements a partial loss function that enables dense tree semantic labeling outcomes from non-dense training samples, and a distance regression complementary task that enforces tree crown boundary constraints and substantially improves the model performance. Our multi-task architecture uses a shared backbone network that learns common representations for both tasks and two task-specific decoders, one for the semantic segmentation output and one for the distance map regression. We report that introducing the complementary task boosts the semantic segmentation performance compared to the single-task counterpart in up to 11% reaching an average user's accuracy of 88.63% and an average producer's accuracy of 88.59%, achieving state-of-art performance for tree species classification in tropical forests.
Tufts University（塔夫茨大学）；University of Cambridge（剑桥大学）；City University of Hong Kong（香港城市大学）；Wake Forest University（维克森林大学）
Ash dieback (Hymenoscyphus fraxineus) is an introduced fungal disease that is causing the widespread death of ash trees across Europe. Remote sensing hyperspectral images encode rich structure that has been exploited for the detection of dieback disease in ash trees using supervised machine learning techniques. However, to understand the state of forest health at landscape-scale, accurate unsupervised approaches are needed. This article investigates the use of the unsupervised Diffusion and VCA-Assisted Image Segmentation (D-VIS) clustering algorithm for the detection of ash dieback disease in…
Label-Efficient 3D Forest Mapping: Self-Supervised and Transfer Learning for Instance Segmentation, Semantic Segmentation, and Species Classification
Helmholtz-Zentrum Dresden-Rossendorf (HZDR), Helmholtz Institute Freiberg for Resource Technology (HIF)（德累斯顿-罗斯托克亥姆霍兹中心（HZDR）、弗里贝格资源技术亥姆霍兹研究所（HIF））；Remote Sensing and Geoinformatics, Freie Universität Berlin（柏林自由大学遥感与地理信息学系）；Institute of Geomatics, BOKU University（博科尼大学测绘学院）；Faculty of Electrical and Computer Engineering, University of Iceland（爱沙尼亚大学电气与计算机工程学院）
SilvaScenes: Tree Detection and Species Classification from Under-Canopy Images in Natural Forests
Northern Robotics Laboratory, Université Laval（北方机器人实验室，拉瓦尔大学）；Département des sciences du bois et de la forêt, Université Laval（林业与木材科学系，拉瓦尔大学）；Institut des Sciences de la Forêt tempérée, Université du Québec en Outaouais（温带森林科学研究所，魁北克outsouais大学）
University of Padova（帕多瓦大学）；Fondazione Edmund Mach（埃德蒙·马赫基金会）；Provincia Autonoma di Trento（特伦托自治省）
This work intends to lay the foundations for identifying the prevailing forest types and the delineation of forest units within private forest inventories in the Autonomous Province of Trento (PAT), using currently available remote sensing solutions. In particular, data from LiDAR and hyperspectral surveys of 2014 made available by PAT were acquired and processed. Such studies are very important in the context of forest management scenarios. The method includes defining tree species ground-truth by outlining single tree crowns with polygons and labeling them. Successively two supervised machin…
There is much current interest in using multi-sensor airborne remote sensing to monitor the structure and biodiversity of forests. This paper addresses the application of non-parametric image registration techniques to precisely align images obtained from multimodal imaging, which is critical for the successful identification of individual trees using object recognition approaches. Non-parametric image registration, in particular the technique of optimizing one objective function containing data fidelity and regularization terms, provides flexible algorithms for image registration. Using a sur…
Bialystok University of Technology（比亚韦斯托克理工大学）；Mammal Research Institute, Polish Academy of Sciences（波兰科学院哺乳动物研究所）
Camera traps are used worldwide to monitor wildlife. Despite the increasing availability of Deep Learning (DL) models, the effective usage of this technology to support wildlife monitoring is limited. This is mainly due to the complexity of DL technology and high computing requirements. This paper presents the implementation of the light-weight and state-of-the-art YOLOv5 architecture for automated labeling of camera trap images of mammals in the Bialowieza Forest (BF), Poland. The camera trapping data were organized and harmonized using TRAPPER software, an open source application for managin…
Point2Tree(P2T) -- framework for parameter tuning of semantic and instance segmentation used with mobile laser scanning data in coniferous forest
Norwegian Institute for Bioeconomy Research (NIBIO)（挪威生物经济研究所（NIBIO））；University College London（伦敦大学学院）；NERC National Centre for Earth Observation（NERC国家地球观测中心）
The collection of a high number of pixel-based labeled training samples for tree species identification is time consuming and costly in operational forestry applications. To address this problem, in this paper we investigate the effectiveness of explanation methods for deep neural networks in performing weakly supervised semantic segmentation using only image-level labels. Specifically, we consider four methods:i) class activation maps (CAM); ii) gradient-based CAM; iii) pixel correlation module; and iv) self-enhancing maps (SEM). We compare these methods with each other using both quantitativ…
University of Cambridge（剑桥大学）；Fondazione E. Mach（埃马努埃莱·马赫基金会）；University of Oxford（牛津大学）；The University of Queensland（昆士兰大学）；Natural England（英国自然署）
Recognising individual trees within remotely sensed imagery has important applications in forest ecology and management. Several algorithms for tree delineation have been suggested, mostly based on locating local maxima or inverted basins in raster canopy height models (CHMs) derived from Light Detection And Ranging (LiDAR) data or photographs. However, these algorithms often lead to inaccurate estimates of forest stand characteristics due to the limited information content of raster CHMs. Here we develop a 3D tree delineation method which uses graph cut to delineate trees from the full 3D LiD…
HyBiomass: Global Hyperspectral Imagery Benchmark Dataset for Evaluating Geospatial Foundation Models in Forest Aboveground Biomass Estimation
Remote Sensing Technology Institute, German Aerospace Center (DLR)（远程传感技术研究所，德国航天中心）；Global Land Monitoring Group, Helmholtz German Research Centre for Geosciences (GFZ)（全球土地监测组，海德堡德国地理科学研究中心）；Department of Aerospace Engineering, University of the Bundeswehr Munich（航空航天工程系，联邦国防军 Munich 大学）；Chair of Data Science in Earth Observation (SiPEO), Technical University of Munich (TUM)（地球观测数据科学教授职位（SiPEO），慕尼黑技术大学）
IBM Research（IBM研究院）；Universidade Federal do Paraná - UFPR（巴拉那联邦大学）
This work focuses on cost reduction methods for forest species recognition systems. Current state-of-the-art shows that the accuracy of these systems have increased considerably in the past years, but the cost in time to perform the recognition of input samples has also increased proportionally. For this reason, in this work we focus on investigating methods for cost reduction locally (at either feature extraction or classification level individually) and globally (at both levels combined), and evaluate two main aspects: 1) the impact in cost reduction, given the proposed measures for it; and …
WoodYOLO: A Novel Object Detector for Wood Species Detection in Microscopic Images
Fraunhofer ITWM（弗劳恩霍夫信息技术与微系统研究所）；Thünen Institute of Wood Research（滕木材研究所）；Institute of Machine Learning and Analysis (IMLA)（机器学习与分析研究所）；Offenburg University（奥芬堡大学）
Remote Sensing Technology Institute (IMF), German Aerospace Center (DLR)（德国航空航天中心遥感技术研究所）；Signal Processing in Earth Observation (SiPEO), Technische Universität München (TUM)（慕尼黑工业大学地球观测信号处理中心）；Faculty of Electrical and Computer Engineering, University of Iceland（冰岛大学电气与计算机工程学院）；Department of Technology of Computers and Communications, University of Extremadura（埃斯特雷马杜拉大学计算机与通信技术系）
In this paper, an approach is proposed to fuse LiDAR and hyperspectral data, which considers both spectral and spatial information in a single framework. Here, an extended self-dual attribute profile (ESDAP) is investigated to extract spatial information from a hyperspectral data set. To extract spectral information, a few well-known classifiers have been used such as support vector machines (SVMs), random forests (RFs), and artificial neural networks (ANNs). The proposed method accurately classify the relatively volumetric data set in a few CPU processing time in a real ill-posed situation wh…
From Articles to Canopies: Knowledge-Driven Pseudo-Labelling for Tree Species Classification using LLM Experts
Institute of Theoretical and Applied Informatics, Polish Academy of Sciences（波兰科学院理论与应用信息学研究所）；University of Lodz（卢布林大学）；University of Warsaw, Faculty of Geography and Regional Studies, Department of Geoinformatics, Cartography and Remote Sensing（华沙大学地理与区域研究学院，地理信息学、制图与遥感系）
Water is a key component of life, the natural environment and human health. For monitoring the conditions of a water body, the chlorophyll a concentration can serve as a proxy for nutrients and oxygen supply. In situ measurements of water quality parameters are often time-consuming, expensive and limited in areal validity. Therefore, we apply remote sensing techniques. During field campaigns, we collected hyperspectral data with a spectrometer and in situ measured chlorophyll a concentrations of 13 inland water bodies with different spectral characteristics. One objective of this study is to e…
University of Cambridge（剑桥大学）；Royal Society for Protection of Birds（英国皇家鸟类保护协会）；Jambi University（占碑大学）；PT Restorasi Ekosistem Indonesia（印度尼西亚生态系统恢复公司）
Logged forests cover four million square kilometres of the tropics and restoring these forests is essential if we are to avoid the worst impacts of climate change, yet monitoring recovery is challenging. Tracking the abundance of visually identifiable, early-successional species enables successional status and thereby restoration progress to be evaluated. Here we present a new pipeline, SLIC-UAV, for processing Unmanned Aerial Vehicle (UAV) imagery to map early-successional species in tropical forests. The pipeline is novel because it comprises: (a) a time-efficient approach for labelling crow…
Research Centre on Interactive Media, Smart Systems and Emerging Technologies (RISE)（互动媒体、智能系统与新兴技术研究中心）；University of Twente（特温特大学）；Leibniz University Hannover（莱布尼茨汉诺威大学）
Natural forests are complex ecosystems whose tree species distribution and their ecosystem functions are still not well understood. Sustainable management of these forests is of high importance because of their significant role in climate regulation, biodiversity, soil erosion and disaster prevention among many other ecosystem services they provide. In Japan particularly, natural forests are mainly located in steep mountains, hence the use of aerial imagery in combination with computer vision are important modern tools that can be applied to forest research. Thus, this study constitutes a prel…
Identification of tree species plays a key role in forestry related tasks like forest conservation, disease diagnosis and plant production. There had been a debate regarding the part of the tree to be used for differentiation, whether it should be leaves, fruits, flowers or bark. Studies have proven that bark is of utmost importance as it will be present despite seasonal variations and provides a characteristic identity to a tree by variations in the structure. In this paper, a deep learning based approach is presented by leveraging the method of computer vision to classify 50 tree species, on…
Soil texture is important for many environmental processes. In this paper, we study the classification of soil texture based on hyperspectral data. We develop and implement three 1-dimensional (1D) convolutional neural networks (CNN): the LucasCNN, the LucasResNet which contains an identity block as residual network, and the LucasCoordConv with an additional coordinates layer. Furthermore, we modify two existing 1D CNN approaches for the presented classification task. The code of all five CNN approaches is available on GitHub (Riese, 2019). We evaluate the performance of the CNN approaches and…
Prospects for Mitigating Spectral Variability in Tropical Species Classification Using Self-Supervised Learning
AMAP；Univ Montpellier（蒙彼利埃大学）；CIRAD（法国农业发展研究国际合作中心）；CNRS（法国国家科学研究中心）；INRAE（法国国家农业、食品与环境研究院）；IRD（法国发展研究院）；Technical University of Munich（慕尼黑工业大学）；German Aerospace Center（德国航空航天中心）；Univ. Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；Inria（法国国家信息与自动化研究所）；Grenoble INP（格勒诺布尔理工学院）；LJK（约瑟夫·傅里叶实验室）
California State Polytechnic University, Pomona（加州州立理工大学波莫纳分校）；University of California, Davis（加州大学戴维斯分校）
The large data size and dimensionality of hyperspectral data demands complex processing and data analysis. Multispectral data do not suffer the same limitations, but are normally restricted to blue, green, red, red edge, and near infrared bands. This study aimed to identify the optimal set of spectral bands for nitrogen detection in grape leaves using ensemble feature selection on hyperspectral data from over 3,000 leaves from 150 Flame Seedless table grapevines. Six machine learning base rankers were included in the ensemble: random forest, LASSO, SelectKBest, ReliefF, SVM-RFE, and chaotic cr…
Deep Learning for Automated Identification of Vietnamese Timber Species: A Tool for Ecological Monitoring and Conservation
Mathematical Modeling Laboratory, Graduate School of Bioresource and Bioenvironmental Sciences, Kyushu University（九州大学生物资源与生物环境科学研究生院数学建模实验室）；International Cooperation Center, Thai Nguyen University（Thai Nguyen大学国际合作中心）；Centrer of Crop Research for Adaptation to Climate Change, Thai Nguyen University of Agriculture and Forestry（Thai Nguyen农业与林业大学适应气候变化作物研究中心）
Skolkovo Institute of Science and Technology（斯科尔科沃科学技术学院）；Ulm University（乌尔姆大学）
For many countries like Russia, Canada, or the USA, a robust and detailed tree species inventory is essential to manage their forests sustainably. Since one can not apply unmanned aerial vehicle (UAV) imagery-based approaches to large-scale forest inventory applications, the utilization of machine learning algorithms on satellite imagery is a rising topic of research. Although satellite imagery quality is relatively low, additional spectral channels provide a sufficient amount of information for tree crown classification tasks. Assuming that tree crowns are detected already, we use embeddings …
SpectralEarth: Training Hyperspectral Foundation Models at Scale
Chair of Data Science in Earth Observation (SiPEO), Technical University of Munich (TUM)（地球观测数据科学教授职位（SiPEO），慕尼黑技术大学）；Remote Sensing Technology Institute (IMF), German Aerospace Center (DLR)（遥感技术研究所（IMF），德国航空航天中心（DLR））；Univ. Grenoble Alpes, Inria, CNRS, Grenoble INP, LJK（格勒诺布尔阿尔卑斯大学，法国国家科学研究中心（CNRS），格勒诺布尔高等理工学院（Grenoble INP），LJK）
IBM Research（IBM研究院）；German Aerospace Center (DLR)（德国航空航天中心（DLR））
Vegetation, trees in particular, sequester carbon by absorbing carbon dioxide from the atmosphere. However, the lack of efficient quantification methods of carbon stored in trees renders it difficult to track the process. We present an approach to estimate the carbon storage in trees based on fusing multi-spectral aerial imagery and LiDAR data to identify tree coverage, geometric shape, and tree species -- key attributes to carbon storage quantification. We demonstrate that tree species information and their three-dimensional geometric shapes can be estimated from aerial imagery in order to de…
IBM T.J. Watson Research Center（IBM T.J. 沃森研究中心）；IBM Research（IBM研究院）
One of the impacts of climate change is the difficulty of tree regrowth after wildfires over areas that traditionally were covered by certain tree species. Here a deep learning model is customized to classify land covers from four-band aerial imagery before and after wildfires to study the prolonged consequences of wildfires on tree species. The tree species labels are generated from manually delineated maps for five land cover classes: Conifer, Hardwood, Shrub, ReforestedTree and Barren land. With an accuracy of $92\%$ on the test split, the model is applied to three wildfires on data from 20…
Tree species identification using bark images is a challenging problem that could prove useful for many forestry related tasks. However, while the recent progress in deep learning showed impressive results on standard vision problems, a lack of datasets prevented its use on tree bark species classification. In this work, we present, and make publicly available, a novel dataset called BarkNet 1.0 containing more than 23,000 high-resolution bark images from 23 different tree species over a wide range of tree diameters. With it, we demonstrate the feasibility of species recognition through bark i…
Tree perception is an essential building block toward autonomous forestry operations. Current developments generally consider input data from lidar sensors to solve forest navigation, tree detection and diameter estimation problems. Whereas cameras paired with deep learning algorithms usually address species classification or forest anomaly detection. In either of these cases, data unavailability and forest diversity restrain deep learning developments for autonomous systems. So, we propose two densely annotated image datasets - 43k synthetic, 100 real - for bounding box, segmentation mask and…
OpenForest: A data catalogue for machine learning in forest monitoring
McGill University（麦吉尔大学）；Mila, Quebec AI Institute（魁北克人工智能研究所（Mila））；Leipzig University（莱比锡大学）；German Centre for Integrative Biodiversity Research (iDiv)（德国综合生物多样性研究中心）；Université de Montréal（蒙特利尔大学）
Understanding Representation Gaps Across Scales in Tropical Tree Species Classification from Drone Imagery
Mila – Quebec AI Institute（魁北克人工智能研究所）；McGill University（麦吉尔大学）；Université de Montréal（蒙特利尔大学）；Cary Institute of Ecosystem Studies（生态系统研究所）；Smithsonian Tropical Research Institute（史密松国家热带研究所）；Department of Plant Sciences, University of Cambridge（剑桥大学植物科学系）；Universidade do Estado do Mato Grosso (UNEMAT)（马托格罗索州立大学（UNEMAT））
Manchester Metropolitan University（曼彻斯特城市大学）；Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）；Brunel University（布鲁内尔大学）
Spectral-spatial based deep learning models have recently proven to be effective in hyperspectral image (HSI) classification for various earth monitoring applications such as land cover classification and agricultural monitoring. However, due to the nature of "black-box" model representation, how to explain and interpret the learning process and the model decision, especially for vegetation classification, remains an open challenge. This study proposes a novel interpretable deep learning model -- a biologically interpretable two-stage deep neural network (BIT-DNN), by incorporating the prior-k…
Benchmarking tree species classification from proximally-sensed laser scanning data: introducing the FOR-species20K dataset
Norwegian Institute for Bioeconomy Research (NIBIO)（挪威生物经济研究所）；University of Cambridge（剑桥大学）；Jan Evangelista Purkyně University in Ústí n. L.（拉贝河畔乌斯季扬·埃万杰利斯塔·普尔基涅大学）；University of Freiburg（弗莱堡大学）；Georg-August-Universität Göttingen（哥廷根大学）；TU Wien（维也纳工业大学）；Swiss Federal Institute for Forest, Snow and Landscape Research WSL（瑞士联邦森林、雪与景观研究所）；University of British Columbia（不列颠哥伦比亚大学）；Ghent University（根特大学）；Heidelberg University（海德堡大学）；University of Eastern Finland（东芬兰大学）；Silva Tarouca Research Institute（席尔瓦·塔劳卡研究所）；Forest Research Institute（森林研究所）；Commonwealth Scientific and Industrial Research Organization (CSIRO)（联邦科学与工业研究组织）；Charles Darwin University（查尔斯·达尔文大学）；University College of London（伦敦大学学院）；Natural Resources Institute Finland (Luke)（芬兰自然资源研究所）；Council for Agricultural Research and Economics（农业研究与经济委员会）；University of Lethbridge（莱斯布里奇大学）；National Research Council – Institute of BioEconomy（国家研究委员会生物经济研究所）；Free University of Bolzano（博尔扎诺自由大学）
Airborne laser scanning (LiDAR) point clouds over large forested areas can be processed to segment individual trees and subsequently extract tree-level information. Existing segmentation procedures typically detect more than 90% of overstory trees, yet they barely detect 60% of understory trees because of the occlusion effect of higher canopy layers. Although understory trees provide limited financial value, they are an essential component of ecosystem functioning by offering habitat for numerous wildlife species and influencing stand development. Here we model the occlusion effect in terms of…
Urban trees help regulate temperature, reduce energy consumption, improve urban air quality, reduce wind speeds, and mitigating the urban heat island effect. Urban trees also play a key role in climate change mitigation and global warming by capturing and storing atmospheric carbon-dioxide which is the largest contributor to greenhouse gases. Automated tree detection and species classification using aerial imagery can be a powerful tool for sustainable forest and urban tree management. Hence, This study first offers a pipeline for generating labelled dataset of urban trees using Google Map's a…
