Unsupervised deep learning for semantic segmentation of multispectral LiDAR forest point clouds
Department of Remote Sensing and Photogrammetry, Finnish Geospatial Research Institute FGI, The National Land Survey of Finland（遥感与摄影测量系，芬兰地理研究 institute FGI，芬兰国家土地测绘局）；D Optical Metrology (3DOM) unit, Bruno Kessler Foundation (FBK)（3D光学计量（3DOM）单元，布鲁诺·凯瑟尔基金会（FBK））；Department of Built Environment, Aalto University, School of Engineering（环境学院，艾尔托大学，工程学院）
Lassi Ruoppa, Oona Oinonen, Josef Taher, Matti Lehtomäki, Narges Takhtkeshha, Antero Kukko, Harri Kaartinen, Juha Hyyppä
备注：
30 pages, 10 figures
发表信息：
ISPRS Journal of Photogrammetry and Remote Sensing, Volume 228, 2025, pages 694-722
Point clouds captured with laser scanning systems from forest environments can be utilized in a wide variety of applications within forestry and plant ecology, such as the estimation of tree stem attributes, leaf angle distribution, and above-ground biomass. However, effectively utilizing the data in such tasks requires the semantic segmentation of the data into wood and foliage points, also known as leaf-wood separation. The traditional approach to leaf-wood separation has been geometry- and radiometry-based unsupervised algorithms, which tend to perform poorly on data captured with airborne laser scanning (ALS) systems, even with a high point density. While recent machine and deep learning approaches achieve great results even on sparse point clouds, they require manually labeled training data, which is often extremely laborious to produce. Multispectral (MS) information has been demonstrated to have potential for improving the accuracy of leaf-wood separation, but quantitative assessment of its effects has been lacking. This study proposes a fully unsupervised deep learning method, GrowSP-ForMS, which is specifically designed for leaf-wood separation of high-density MS ALS point clouds and based on the GrowSP architecture. GrowSP-ForMS achieved a mean accuracy of 84.3% and a mean intersection over union (mIoU) of 69.6% on our MS test set, outperforming the unsupervised reference methods by a significant margin. When compared to supervised deep learning methods, our model performed similarly to the slightly older PointNet architecture but was outclassed by more recent approaches. Finally, two ablation studies were conducted, which demonstrated that our proposed changes increased the test set mIoU of GrowSP-ForMS by 29.4 percentage points (pp) in comparison to the original GrowSP model and that utilizing MS data improved the mIoU by 5.6 pp from the monospectral case.
3D LULC classification using multispectral LiDAR and deep learning: current and prospective schemes
D Optical Metrology (3DOM) Unit, Bruno Kessler Foundation (FBK)（3D光学计量（3DOM）单元，布鲁诺·凯塞尔基金会（FBK））；Department of Geodesy and Geoinformation, TU Wien（测绘与地理信息系，维也纳技术大学）；Helmholtz-ZentrumDresden-Rossendorf (HZDR), Helmholtz Institute Freiberg for Resource Technology (HIF)（德累斯顿-罗斯托克赫尔姆霍尔茨研究中心（HZDR），资源技术赫尔姆霍尔茨研究所（HIF））；Freie Universität Berlin, Remote Sensing and Geoinformatics（柏林自由大学，遥感与地理信息学）；Department of Remote Sensing and Photogrammetry, Finnish Geospatial Research Institute FGI, The National Land Survey of Finland（芬兰地理研究所（FGI），芬兰国家土地测绘局）
3D forest semantic segmentation using multispectral LiDAR and 3D deep learning
Bruno Kessler Foundation (FBK)（布鲁诺·凯塞勒基金会）；TU Wien（维也纳工业大学）；Finnish Geospatial Research Institute（芬兰地理空间研究所）；National Land Survey of Finland（芬兰国家土地测量局）
Riverine Land Cover Mapping through Semantic Segmentation of Multispectral Point Clouds
Department of Remote Sensing and Photogrammetry, Finnish Geospatial Research Institute FGI（遥感与摄影测量系，芬兰地理研究所）；Faculty of Technology, University of Turku（技术学院，图尔库大学）；Department of Geography and Geology, University of Turku（地理与地质系，图尔库大学）；Faculty of Geological Sciences, Complutense University of Madrid（地质科学学院，马德里卡洛斯三世大学）
SelectAnyTree: A Promptable Instance Segmentation Model for 3D Forest LiDAR Point Clouds
Nagoya University（名古屋大学）；RIKEN（理化学研究所）；University of Freiburg（弗莱堡大学）；University of California, Los Angeles（加州大学洛杉矶分校）；University of Twente（特温特大学）；KC Machine Learning Lab（KC机器学习实验室）；Hanoi University of Science and Technology（河内科技大学）；Ritsumeikan University（立命馆大学）
University of Pennsylvania（宾夕法尼亚大学）；University of Sao Paulo（圣保罗大学）
This paper describes an end-to-end pipeline for tree diameter estimation based on semantic segmentation and lidar odometry and mapping. Accurate mapping of this type of environment is challenging since the ground and the trees are surrounded by leaves, thorns and vines, and the sensor typically experiences extreme motion. We propose a semantic feature based pose optimization that simultaneously refines the tree models while estimating the robot pose. The pipeline utilizes a custom virtual reality tool for labeling 3D scans that is used to train a semantic segmentation network. The masked point…
Learning Image-based Tree Crown Segmentation from Enhanced Lidar-based Pseudo-labels
Department of Remote Sensing and Photogrammetry, Finnish Geospatial Research Institute（遥感与摄影测量系，芬兰地理研究机构）；Department of Computer Science, Aalto University（计算机科学系，阿alto大学）
German Aerospace Center (DLR)（德国航空航天中心）；Technical University of Munich (TUM)（慕尼黑工业大学）；Univ. Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；Inria（法国国家信息与自动化研究所）；CNRS（法国国家科学研究中心）；Grenoble INP（格勒诺布尔国立理工学院）；University of Iceland（冰岛大学）；RIKEN（日本理化学研究所）
Due to the ever-growing diversity of the data source, multi-modality feature learning has attracted more and more attention. However, most of these methods are designed by jointly learning feature representation from multi-modalities that exist in both training and test sets, yet they are less investigated in absence of certain modality in the test phase. To this end, in this letter, we propose to learn a shared feature space across multi-modalities in the training process. By this way, the out-of-sample from any of multi-modalities can be directly projected onto the learned space for a more e…
Semantic segmentation of forest stands using deep learning
Faculty of Environmental Sciences and Natural Resource Management, Norwegian University of Life Sciences（生命科学大学环境科学与自然资源管理学院）；Faculty of Science and Technology, Norwegian University of Life Sciences（生命科学大学科学与技术学院）
Unsupervised semantic segmentation of urban high-density multispectral point clouds
Finnish Geospatial Research Institute FGI, the National Land Survey of Finland（芬兰地理空间研究所（FGI，芬兰国家土地测量局））；Department of Built Environment（建筑环境系）；School of Engineering（工程学院）；Aalto University（阿尔托大学）
本文提出无监督地面感知深度聚类方法 GroupSP，通过超点分组与伪标签迭代训练，实现高密度多光谱 ALS 城市点云的七类语义分割，并以极少标注取得高精度。
UiT The Arctic University of Norway（挪威北极大学）；NORCE Norwegian Research Centre（挪威NORCE研究中心）；Norwegian Institute for Nature Research（挪威自然研究所）
Several generic methods have recently been developed for change detection in heterogeneous remote sensing data, such as images from synthetic aperture radar (SAR) and multispectral radiometers. However, these are not well suited to detect weak signatures of certain disturbances of ecological systems. To resolve this problem we propose a new approach based on image-to-image translation and one-class classification (OCC). We aim to map forest mortality caused by an outbreak of geometrid moths in a sparsely forested forest-tundra ecotone using multisource satellite images. The images preceding an…
FoMo: Multi-Modal, Multi-Scale and Multi-Task Remote Sensing Foundation Models for Forest Monitoring
National Observatory of Athens（雅典国家天文台）；National Technical University of Athens（雅典国家技术大学）；Harokopio University of Athens（哈罗科皮奥大学）；McGill University（麦吉尔大学）
The simultaneous capture of imaging data at multiple wavelengths across the electromagnetic spectrum is highly challenging, requiring complex and costly multispectral image sensors. In this study, we introduce a comprehensive framework for performing simultaneous multispectral imaging using conventional image sensors with color filter arrays via numerical demultiplexing of the color image sensor measurements. A numerical forward model characterizing the formation of sensor measurements from light spectra hitting the sensor is constructed based on a comprehensive spectral characterization of th…
Multispectral LiDAR data for extracting tree points in urban and suburban areas
Department of Geodesy and Geoinformation, TU Wien（大地测量与地理信息系，维也纳技术大学）；D Optical Metrology (3DOM) unit, Bruno Kessler Foundation (FBK)（三维光学计量（3DOM）单元，布鲁诺·凯斯勒基金会）；Department of Photogrammetry and Remote Sensing, Finnish Geospatial Research Institute, National Land Survey of Finland（摄影测量与遥感系，芬兰地理研究 institute，芬兰国家土地测绘局）
In this article, we evaluate unsupervised anomaly detection methods in multispectral images obtained with a wavelength-independent synthetic aperture sensing technique, called Airborne Optical Sectioning (AOS). With a focus on search and rescue missions that apply drones to locate missing or injured persons in dense forest and require real-time operation, we evaluate runtime vs. quality of these methods. Furthermore, we show that color anomaly detection methods that normally operate in the visual range always benefit from an additional far infrared (thermal) channel. We also show that, even wi…
Remote Sensing Technology Institute, German Aerospace Center（德国航空航天中心遥感技术研究所）；Technical University of Munich（慕尼黑工业大学）；TJ Watson Research Center, IBM Research（IBM研究院托马斯·J·沃森研究中心）
We present and evaluate a weakly-supervised methodology to quantify the spatio-temporal distribution of urban forests based on remotely sensed data with close-to-zero human interaction. Successfully training machine learning models for semantic segmentation typically depends on the availability of high-quality labels. We evaluate the benefit of high-resolution, three-dimensional point cloud data (LiDAR) as source of noisy labels in order to train models for the localization of trees in orthophotos. As proof of concept we sense Hurricane Sandy's impact on urban forests in Coney Island, New York…
We present a method for detecting and mapping trees in noisy stereo camera point clouds, using a learned 3-D object detector. Inspired by recent advancements in 3-D object detection using a pseudo-lidar representation for stereo data, we train a PointRCNN detector to recognize trees in forest-like environments. We generate detector training data with a novel automatic labeling process that clusters a fused global point cloud. This process annotates large stereo point cloud training data sets with minimal user supervision, and unlike previous pseudo-lidar detection pipelines, requires no 3-D gr…
ForCM: Forest Cover Mapping from Multispectral Sentinel-2 Image by Integrating Deep Learning with Object-Based Image Analysis
BRAC University（布拉克大学）；Charles Sturt University（查尔斯·斯特劳特大学）；University of New England（新英格兰大学）；University of Technology Sydney（技术悉尼大学）；Griffith University（格里菲斯大学）；Universiti Kebangsaan Malaysia (UKM)（马来亚大学）；Federation University Australia（联邦大学澳大利亚）；RMIT University（皇家墨尔本理工大学）
University of Padova（帕多瓦大学）；Fondazione Edmund Mach（埃德蒙·马赫基金会）；Provincia Autonoma di Trento（特伦托自治省）
This work intends to lay the foundations for identifying the prevailing forest types and the delineation of forest units within private forest inventories in the Autonomous Province of Trento (PAT), using currently available remote sensing solutions. In particular, data from LiDAR and hyperspectral surveys of 2014 made available by PAT were acquired and processed. Such studies are very important in the context of forest management scenarios. The method includes defining tree species ground-truth by outlining single tree crowns with polygons and labeling them. Successively two supervised machin…
Nagoya University（名古屋大学）；Chubu University（中部大学）；Tokyo Institute of Technology（东京工业大学）；Advanced Industrial Science and Technology（产业技术综合研究所）
In this paper, we propose a method for cloud removal from visible light RGB satellite images by extending the conditional Generative Adversarial Networks (cGANs) from RGB images to multispectral images. Satellite images have been widely utilized for various purposes, such as natural environment monitoring (pollution, forest or rivers), transportation improvement and prompt emergency response to disasters. However, the obscurity caused by clouds makes it unstable to monitor the situation on the ground with the visible light camera. Images captured by a longer wavelength are introduced to reduce…
NormalView: tree species classification from backpack and aerial lidar data using geometric projections
Finnish Geospatial Research Institute FGI（芬兰地理研究 institute FGI）；The National Land Survey of Finland（芬兰国家土地调查局）；Department of Remote Sensing and Photogrammetry（遥感与摄影测量系）；Department of Built Environment, School of Engineering（环境系，工程学院）
The Hong Kong Polytechnic University（香港理工大学）；The Hong Kong Polytechnic University Shenzhen Research Institute（香港理工大学深圳研究院）；Nanjing Forestry University（南京林业大学）
Forest land plays a vital role in global climate, ecosystems, farming and human living environments. Therefore, forest biomass estimation methods are necessary to monitor changes in the forest structure and function, which are key data in natural resources research. Although accurate forest biomass measurements are important in forest inventory and assessments, high-density measurements that involve airborne light detection and ranging (LiDAR) at a low flight height in large mountainous areas are highly expensive. The objective of this study was to quantify the aboveground biomass (AGB) of a p…
There is much current interest in using multi-sensor airborne remote sensing to monitor the structure and biodiversity of forests. This paper addresses the application of non-parametric image registration techniques to precisely align images obtained from multimodal imaging, which is critical for the successful identification of individual trees using object recognition approaches. Non-parametric image registration, in particular the technique of optimizing one objective function containing data fidelity and regularization terms, provides flexible algorithms for image registration. Using a sur…
TreeLearn: A deep learning method for segmenting individual trees from ground-based LiDAR forest point clouds
University of Göttingen（哥廷根大学）；Max Planck Institute for Dynamics and Self-Organization（马克斯·普朗克动力学与自组织研究所）；Campus Institute Data Science, University of Göttingen（哥廷根大学校园数据科学研究所）
Minkowski-MambaNet: A Point Cloud Framework with Selective State Space Models for Forest Biomass Quantification
School of Computer Science and Technology, the Key Laboratory of Signal Detection and Processing, and the Xinjiang Multimodal Intelligent Processing and Information Security Engineering Technology Research Center, Xinjiang University（计算机科学与技术学院、信号检测与处理重点实验室以及新疆多模态智能处理与信息安全工程技术研究中心，新疆大学）；National Key Laboratory for Novel Software Technology, Nanjing University（新型软件技术国家实验室，南京大学）；Xinjiang Guanghui New Energy Co., Ltd（新疆广汇新能源有限公司）；Hongyousoft Co., Ltd（宏远软件有限公司）
ForestFormer3D: A Unified Framework for End-to-End Segmentation of Forest LiDAR 3D Point Clouds
Norwegian Institute of Bioeconomy Research (NIBIO)（挪威生物经济研究所）；Silva Tarouca Research Institute for Landscape and Ornamental Gardening（Silva Tarouca景观与园艺研究 institute）
School of Geospatial Engineering and Science, Sun Yat-Sen University（中山大学测绘科学与技术学院）；Department of Land Surveying and Geo-Informatics, The Hong Kong Polytechnic University（香港理工大学土地测量及地理资讯学系）；Changjiang River Scientific Research Institute (CRSRI)（长江水利委员会长江科学院）；Key Laboratory of Digital Earth Science, Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院数字地球重点实验室）；Southern Marine Science and Engineering Guangdong Laboratory (Zhuhai)（南方海洋科学与工程广东省实验室（珠海））
Registration of unmanned aerial vehicle laser scanning (ULS) and ground light detection and ranging (LiDAR) point clouds in forests is critical to create a detailed representation of a forest structure and an accurate inversion of forest parameters. However, forest occlusion poses challenges for marker-based registration methods, and some marker-free automated registration methods have low efficiency due to the process of object (e.g., tree, crown) segmentation. Therefore, we use a divide-and-conquer strategy and propose an automated and efficient method to register ULS and ground LiDAR point …
Benchmarking individual tree segmentation using multispectral airborne laser scanning data: the FGI-EMIT dataset
Finnish Geospatial Research Institute FGI（芬兰地理空间研究 institute）；The National Land Survey of Finland（芬兰国家土地调查局）；Department of Remote Sensing and Photogrammetry（遥感与摄影测量系）；Department of Built Environment, School of Engineering, Aalto University（环境学院，工程学院，阿alto大学）
High Resolution Tree Height Mapping of the Amazon Forest using Planet NICFI Images and LiDAR-Informed U-Net Model
CTrees；Jet Propulsion Laboratory, California Institute of Technology（加州理工学院喷气推进实验室）；Institute of Environment and Sustainability, University of California, Los Angeles（加州大学洛杉矶分校环境与可持续发展研究所）；Quapá Lab, Faculty of Architecture and Urbanism, University of São Paulo（圣保罗大学建筑与城市规划学院Quapá实验室）；Gamma Remote Sensing Ag（Gamma遥感公司）；USDA Forest Service, International Institute of Tropical Forestry（美国农业部林业局热带林业研究所）；EMBRAPA Satellite Monitoring（巴西农业研究院卫星监测中心）；Remote Sensing Division, National Institute for Space Research—INPE（巴西国家空间研究院遥感部）；Department of Geosciences and Natural Resource Management, University of Copenhagen（哥本哈根大学地球科学与自然资源管理系）；Laboratoire des Sciences du Climat et de l’Environnement, CEA-CNRS-UVSQ（气候与环境科学实验室（CEA-CNRS-UVSQ））
TreeLoc: 6-DoF LiDAR Global Localization in Forests via Inter-Tree Geometric Matching
Dept. of Mechanical Engineering, SNU（机械工程系，首尔国立大学）；Oxford Robotics Institute, University of Oxford（牛津大学机器人研究所）；CSIRO Robotics, DATA61, CSIRO and the School of Electrical Engineering and Computer Science, The University of Queensland (UQ)（CSIRO机器人、DATA61、CSIRO及昆士兰大学电气工程与计算机科学学院）
Airborne LiDAR point cloud representing a forest contains 3D data, from which vertical stand structure even of understory layers can be derived. This paper presents a tree segmentation approach for multi-story stands that stratifies the point cloud to canopy layers and segments individual tree crowns within each layer using a digital surface model based tree segmentation method. The novelty of the approach is the stratification procedure that separates the point cloud to an overstory and multiple understory tree canopy layers by analyzing vertical distributions of LiDAR points within overlappi…
This paper presents a non-parametric approach for segmenting trees from airborne LiDAR data in deciduous forests. Based on the LiDAR point cloud, the approach collects crown information such as steepness and height on-the-fly to delineate crown boundaries, and most importantly, does not require a priori assumptions of crown shape and size. The approach segments trees iteratively starting from the tallest within a given area to the smallest until all trees have been segmented. To evaluate its performance, the approach was applied to the University of Kentucky Robinson Forest, a deciduous closed…
A Deep Learning Approach to Estimate Canopy Height and Uncertainty by Integrating Seasonal Optical, SAR and Limited GEDI LiDAR Data over Northern Forests
Forest canopy height estimation from satellite RGB imagery using large-scale airborne LiDAR-derived training data and monocular depth estimation
Beijing Normal University（北京师范大学）；CSIRO Environment（英联邦科学与工业研究组织环境部）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Wuhan University（武汉大学）；Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）
Multispectral airborne laser scanning for tree species classification: a benchmark of machine learning and deep learning algorithms
Department of Remote Sensing and Photogrammetry, Finnish Geospatial Research Institute FGI（遥感与摄影测量系，芬兰地理研究机构FGI）；Department of Forest Sciences, University of Helsinki（森林科学系，赫尔辛基大学）
Task Planning Support for Arborists and Foresters: Comparing Deep Learning Approaches for Tree Inventory and Tree Vitality Assessment Based on UAV-Data
Multispectral airborne laser scanning dataset for tree species classification: MS-ALS-SPECIES
Department of Remote Sensing and Photogrammetry（遥感与摄影测量系）；Finnish Geospatial Research Institute（芬兰地理研究 institute）；FGI；The National Land Survey of Finland（芬兰国家土地测绘局）
University of Cambridge（剑桥大学）；Fondazione E. Mach（埃马努埃莱·马赫基金会）；University of Oxford（牛津大学）；The University of Queensland（昆士兰大学）；Natural England（英国自然署）
Recognising individual trees within remotely sensed imagery has important applications in forest ecology and management. Several algorithms for tree delineation have been suggested, mostly based on locating local maxima or inverted basins in raster canopy height models (CHMs) derived from Light Detection And Ranging (LiDAR) data or photographs. However, these algorithms often lead to inaccurate estimates of forest stand characteristics due to the limited information content of raster CHMs. Here we develop a 3D tree delineation method which uses graph cut to delineate trees from the full 3D LiD…
Event Spectroscopy: Event-based Multispectral and Depth Sensing using Structured Light
Environmental Robotics Laboratory, Dep. of Environmental Systems Science, ETH Zurich（环境机器人实验室，环境系统科学系，苏黎世联邦理工学院）；Swiss Federal Institute for Forest, Snow and Landscape Research (WSL)（瑞士森林、雪和景观研究联邦研究所）；Robotics and Perception Group, University of Zurich（机器人与感知小组，苏黎世大学）
Airborne discrete return light detection and ranging (LiDAR) point clouds covering forested areas can be processed to segment individual trees and retrieve their morphological attributes. Segmenting individual trees in natural deciduous forests however remained a challenge because of the complex and multi-layered canopy. In this chapter, we present (i) a robust segmentation method that avoids a priori assumptions about the canopy structure, (ii) a vertical canopy stratification procedure that improves segmentation of understory trees, (iii) an occlusion model for estimating the point density o…
