3D forest semantic segmentation using multispectral LiDAR and 3D deep learning
Bruno Kessler Foundation (FBK)（布鲁诺·凯塞勒基金会）；TU Wien（维也纳工业大学）；Finnish Geospatial Research Institute（芬兰地理空间研究所）；National Land Survey of Finland（芬兰国家土地测量局）
Narges Takhtkeshha, Lauris Bocaux, Lassi Ruoppa, Fabio Remondino, Gottfried Mandlburger, Antero Kukko, Juha Hyyppä
Conservation and decision-making regarding forest resources necessitate regular forest inventory. Light detection and ranging (LiDAR) in laser scanning systems has gained significant attention over the past two decades as a remote and non-destructive solution to streamline the labor-intensive and time-consuming procedure of forest inventory. Advanced multispectral (MS) LiDAR systems simultaneously acquire three-dimensional (3D) spatial and spectral information across multiple wavelengths of the electromagnetic spectrum. Consequently, MS-LiDAR technology enables the estimation of both the biochemical and biophysical characteristics of forests. Forest component segmentation is crucial for forest inventory. The synergistic use of spatial and spectral laser information has proven to be beneficial for achieving precise forest semantic segmentation. Thus, this study aims to investigate the potential of MS-LiDAR data, captured by the HeliALS system, providing high-density multispectral point clouds to segment forests into six components: ground, low vegetation, trunks, branches, foliage, and woody debris. Three point-wise 3D deep learning models and one machine learning model, including kernel point convolution, superpoint transformer, point transformer V3, and random forest, are implemented. Our experiments confirm the superior accuracy of the KPConv model. Additionally, various geometric and spectral feature vector scenarios are examined. The highest accuracy is achieved by feeding all three wavelengths (1550 nm, 905 nm, and 532 nm) as the initial features into the deep learning model, resulting in improvements of 33.73% and 32.35% in mean intersection over union (mIoU) and in mean accuracy (mAcc), respectively. This study highlights the excellent potential of multispectral LiDAR for improving the accuracy in fully automated forest component segmentation.
3D LULC classification using multispectral LiDAR and deep learning: current and prospective schemes
D Optical Metrology (3DOM) Unit, Bruno Kessler Foundation (FBK)（3D光学计量（3DOM）单元，布鲁诺·凯塞尔基金会（FBK））；Department of Geodesy and Geoinformation, TU Wien（测绘与地理信息系，维也纳技术大学）；Helmholtz-ZentrumDresden-Rossendorf (HZDR), Helmholtz Institute Freiberg for Resource Technology (HIF)（德累斯顿-罗斯托克赫尔姆霍尔茨研究中心（HZDR），资源技术赫尔姆霍尔茨研究所（HIF））；Freie Universität Berlin, Remote Sensing and Geoinformatics（柏林自由大学，遥感与地理信息学）；Department of Remote Sensing and Photogrammetry, Finnish Geospatial Research Institute FGI, The National Land Survey of Finland（芬兰地理研究所（FGI），芬兰国家土地测绘局）
Unsupervised deep learning for semantic segmentation of multispectral LiDAR forest point clouds
Department of Remote Sensing and Photogrammetry, Finnish Geospatial Research Institute FGI, The National Land Survey of Finland（遥感与摄影测量系，芬兰地理研究 institute FGI，芬兰国家土地测绘局）；D Optical Metrology (3DOM) unit, Bruno Kessler Foundation (FBK)（3D光学计量（3DOM）单元，布鲁诺·凯瑟尔基金会（FBK））；Department of Built Environment, Aalto University, School of Engineering（环境学院，艾尔托大学，工程学院）
Riverine Land Cover Mapping through Semantic Segmentation of Multispectral Point Clouds
Department of Remote Sensing and Photogrammetry, Finnish Geospatial Research Institute FGI（遥感与摄影测量系，芬兰地理研究所）；Faculty of Technology, University of Turku（技术学院，图尔库大学）；Department of Geography and Geology, University of Turku（地理与地质系，图尔库大学）；Faculty of Geological Sciences, Complutense University of Madrid（地质科学学院，马德里卡洛斯三世大学）
University of Pennsylvania（宾夕法尼亚大学）；University of Sao Paulo（圣保罗大学）
This paper describes an end-to-end pipeline for tree diameter estimation based on semantic segmentation and lidar odometry and mapping. Accurate mapping of this type of environment is challenging since the ground and the trees are surrounded by leaves, thorns and vines, and the sensor typically experiences extreme motion. We propose a semantic feature based pose optimization that simultaneously refines the tree models while estimating the robot pose. The pipeline utilizes a custom virtual reality tool for labeling 3D scans that is used to train a semantic segmentation network. The masked point…
SelectAnyTree: A Promptable Instance Segmentation Model for 3D Forest LiDAR Point Clouds
Nagoya University（名古屋大学）；RIKEN（理化学研究所）；University of Freiburg（弗莱堡大学）；University of California, Los Angeles（加州大学洛杉矶分校）；University of Twente（特温特大学）；KC Machine Learning Lab（KC机器学习实验室）；Hanoi University of Science and Technology（河内科技大学）；Ritsumeikan University（立命馆大学）
3D single-photon LiDAR imaging plays an important role in numerous applications. However, long acquisition times and significant data volumes present a challenge to LiDAR imaging. This paper proposes a task-optimized adaptive sampling framework that enables fast acquisition and processing of high-dimensional single-photon LiDAR data. Given a task of interest, the iterative sampling strategy targets the most informative regions of a scene which are defined as those minimizing parameter uncertainties. The task is performed by considering a Bayesian model that is carefully built to allow fast per…
3D Lidar imaging can be a challenging modality when using multiple wavelengths, or when imaging in high noise environments (e.g., imaging through obscurants). This paper presents a hierarchical Bayesian algorithm for the robust reconstruction of multispectral single-photon Lidar data in such environments. The algorithm exploits multi-scale information to provide robust depth and reflectivity estimates together with their uncertainties to help with decision making. The proposed weight-based strategy allows the use of available guide information that can be obtained by using state-of-the-art lea…
Learning Image-based Tree Crown Segmentation from Enhanced Lidar-based Pseudo-labels
Department of Remote Sensing and Photogrammetry, Finnish Geospatial Research Institute（遥感与摄影测量系，芬兰地理研究机构）；Department of Computer Science, Aalto University（计算机科学系，阿alto大学）
In this paper, we present a novel Bayesian approach for estimating spectral and range profiles from single-photon Lidar waveforms associated with single surfaces in the photon-limited regime. In contrast to classical multispectral Lidar signals, we consider a single Lidar waveform per pixel, whereby a single detector is used to acquire information simultaneously at multiple wavelengths. A new observation model based on a mixture of distributions is developed. It relates the unknown parameters of interest to the observed waveforms containing information from multiple wavelengths. Adopting a Bay…
UiT The Arctic University of Norway（挪威北极大学）；NORCE Norwegian Research Centre（挪威NORCE研究中心）；Norwegian Institute for Nature Research（挪威自然研究所）
Several generic methods have recently been developed for change detection in heterogeneous remote sensing data, such as images from synthetic aperture radar (SAR) and multispectral radiometers. However, these are not well suited to detect weak signatures of certain disturbances of ecological systems. To resolve this problem we propose a new approach based on image-to-image translation and one-class classification (OCC). We aim to map forest mortality caused by an outbreak of geometrid moths in a sparsely forested forest-tundra ecotone using multisource satellite images. The images preceding an…
We present a method for detecting and mapping trees in noisy stereo camera point clouds, using a learned 3-D object detector. Inspired by recent advancements in 3-D object detection using a pseudo-lidar representation for stereo data, we train a PointRCNN detector to recognize trees in forest-like environments. We generate detector training data with a novel automatic labeling process that clusters a fused global point cloud. This process annotates large stereo point cloud training data sets with minimal user supervision, and unlike previous pseudo-lidar detection pipelines, requires no 3-D gr…
Semantic segmentation of forest stands using deep learning
Faculty of Environmental Sciences and Natural Resource Management, Norwegian University of Life Sciences（生命科学大学环境科学与自然资源管理学院）；Faculty of Science and Technology, Norwegian University of Life Sciences（生命科学大学科学与技术学院）
Unsupervised semantic segmentation of urban high-density multispectral point clouds
Finnish Geospatial Research Institute FGI, the National Land Survey of Finland（芬兰地理空间研究所（FGI，芬兰国家土地测量局））；Department of Built Environment（建筑环境系）；School of Engineering（工程学院）；Aalto University（阿尔托大学）
本文提出无监督地面感知深度聚类方法 GroupSP，通过超点分组与伪标签迭代训练，实现高密度多光谱 ALS 城市点云的七类语义分割，并以极少标注取得高精度。
The Hong Kong Polytechnic University（香港理工大学）；The Hong Kong Polytechnic University Shenzhen Research Institute（香港理工大学深圳研究院）；Nanjing Forestry University（南京林业大学）
Forest land plays a vital role in global climate, ecosystems, farming and human living environments. Therefore, forest biomass estimation methods are necessary to monitor changes in the forest structure and function, which are key data in natural resources research. Although accurate forest biomass measurements are important in forest inventory and assessments, high-density measurements that involve airborne light detection and ranging (LiDAR) at a low flight height in large mountainous areas are highly expensive. The objective of this study was to quantify the aboveground biomass (AGB) of a p…
FoMo: Multi-Modal, Multi-Scale and Multi-Task Remote Sensing Foundation Models for Forest Monitoring
National Observatory of Athens（雅典国家天文台）；National Technical University of Athens（雅典国家技术大学）；Harokopio University of Athens（哈罗科皮奥大学）；McGill University（麦吉尔大学）
We present a procedure for assessing the urban exposure and seismic vulnerability that integrates LiDAR data with aerial and satellite images. It comprises three phases: first, we segment the satellite image to divide the study area into different urban patterns. Second, we extract building footprints and attributes that represent the type of building of each urban pattern. Finally, we assign the seismic vulnerability to each building using different machine-learning techniques: Decision trees, SVM, logistic regression and Bayesian networks. We apply the procedure to 826 buildings in the city …
Remote Sensing Technology Institute, German Aerospace Center（德国航空航天中心遥感技术研究所）；Technical University of Munich（慕尼黑工业大学）；TJ Watson Research Center, IBM Research（IBM研究院托马斯·J·沃森研究中心）
We present and evaluate a weakly-supervised methodology to quantify the spatio-temporal distribution of urban forests based on remotely sensed data with close-to-zero human interaction. Successfully training machine learning models for semantic segmentation typically depends on the availability of high-quality labels. We evaluate the benefit of high-resolution, three-dimensional point cloud data (LiDAR) as source of noisy labels in order to train models for the localization of trees in orthophotos. As proof of concept we sense Hurricane Sandy's impact on urban forests in Coney Island, New York…
While mobile LiDAR sensors are increasingly used to scan in ecology and forestry applications, reconstruction and characterisation are typically carried out offline (to the best of our knowledge). Motivated by this, we present an online LiDAR system which can run on a handheld device to segment and track individual trees and identify them in a fixed coordinate system. Segments relating to each tree are accumulated over time, and tree models are completed as more scans are captured from different perspectives. Using this reconstruction we then fit a cylinder model to each tree trunk by solving …
A multispectral image camera captures image data within specific wavelength ranges in narrow wavelength bands across the electromagnetic spectrum. Images from a multispectral camera can extract additional information that the human eye or a normal camera fails to capture and thus may have important applications in precision agriculture, forestry, medicine and object identification. Conventional multispectral cameras are made up of multiple image sensors each fitted with a narrow passband wavelength filter and optics, which makes them heavy, bulky, power hungry and very expensive. The multiple …
SALPA: Spaceborne LiDAR Point Adjustment for Enhanced GEDI Footprint Geolocation
Graduate School of Science & Engineering, Saitama University, Japan（埼玉大学理学系研究科）；Japan Aerospace Exploration Agency, Japan（日本宇宙航空研究开发机构）；Graduate School of Horticulture, Chiba University, Japan（千叶大学园艺学系研究科）
Tree species classification at the pixel-level using deep learning and multispectral time series in an imbalanced context
Université of Orléans（奥尔良大学）；INRAE（法国国家农业食品与环境研究院）；CESBIO（环境生物学与生态学研究中心）；Université de Toulouse（图卢兹大学）；CNES（法国国家空间研究中心）；CNRS（法国国家科学研究中心）；IRD（法国发展研究院）；UT3-Paul Sabatier（保罗·萨巴蒂埃第三大学）
Comparing remote sensing-based forest biomass mapping approaches using new forest inventory plots in contrasting forests in northeastern and southwestern China
School of GeoSciences, University of Edinburgh（爱丁堡大学地球科学学院）；Space Intelligence Ltd（空间智能有限公司）；Finnish Geospatial Research Institute（芬兰地理空间研究所）；Institute for Logic, Language and Computation, University of Amsterdam（阿姆斯特丹大学逻辑、语言与计算研究所）；Department of Forest Science, University of Helsinki（赫尔辛基大学森林科学系）；Jihua Lab（季华实验室）
