There is much current interest in using multi-sensor airborne remote sensing to monitor the structure and biodiversity of forests. This paper addresses the application of non-parametric image registration techniques to precisely align images obtained from multimodal imaging, which is critical for the successful identification of individual trees using object recognition approaches. Non-parametric image registration, in particular the technique of optimizing one objective function containing data fidelity and regularization terms, provides flexible algorithms for image registration. Using a survey of woodlands in southern Spain as an example, we show that non-parametric image registration can be successful at fusing datasets when there is little prior knowledge about how the datasets are interrelated (i.e. in the absence of ground control points). The validity of non-parametric registration methods in airborne remote sensing is demonstrated by a series of experiments. Precise data fusion is a prerequisite to accurate recognition of objects within airborne imagery, so non-parametric image registration could make a valuable contribution to the analysis pipeline.
Nowadays, the hyperspectral remote sensing imagery HSI becomes an important tool to observe the Earth's surface, detect the climatic changes and many other applications. The classification of HSI is one of the most challenging tasks due to the large amount of spectral information and the presence of redundant and irrelevant bands. Although great progresses have been made on classification techniques, few studies have been done to provide practical guidelines to determine the appropriate classifier for HSI. In this paper, we investigate the performance of four supervised learning algorithms, na…
Hyperspectral imaging offers new perspectives for diverse applications, ranging from the monitoring of the environment using airborne or satellite remote sensing, precision farming, food safety, planetary exploration, or astrophysics. Unfortunately, the spectral diversity of information comes at the expense of various sources of degradation, and the lack of accurate ground-truth "clean" hyperspectral signals acquired on the spot makes restoration tasks challenging. In particular, training deep neural networks for restoration is difficult, in contrast to traditional RGB imaging problems where d…
Helmholtz AI（亥姆霍兹人工智能平台）；Karlsruhe Institute of Technology (KIT)（卡尔斯鲁厄理工学院）；Helmholtz-Zentrum Dresden-Rossendorf (HZDR)（亥姆霍兹德累斯顿罗森多夫研究中心）
As with any physical instrument, hyperspectral cameras induce different kinds of noise in the acquired data. Therefore, Hyperspectral denoising is a crucial step for analyzing hyperspectral images (HSIs). Conventional computational methods rarely use GPUs to improve efficiency and are not fully open-source. Alternatively, deep learning-based methods are often open-source and use GPUs, but their training and utilization for real-world applications remain non-trivial for many researchers. Consequently, we propose HyDe: the first open-source, GPU-accelerated Python-based, hyperspectral image deno…
Local spectral similarity (LSS) algorithm has been developed for detecting homogeneous areas and edges in hyperspectral images (HSIs). The proposed algorithm transforms the 3-D data cube (within a spatial window) into a spectral similarity matrix by calculating the vector-similarity between the center pixel-spectrum and the neighborhood spectra. The final edge intensity is derived upon order statistics of the similarity matrix or spatial convolution of the similarity matrix with the spatial kernels. The LSS algorithm facilitates simultaneous use of spectral-spatial information for the edge det…
University of Cambridge（剑桥大学）；Fondazione E. Mach（埃马努埃莱·马赫基金会）；University of Oxford（牛津大学）；The University of Queensland（昆士兰大学）；Natural England（英国自然署）
Recognising individual trees within remotely sensed imagery has important applications in forest ecology and management. Several algorithms for tree delineation have been suggested, mostly based on locating local maxima or inverted basins in raster canopy height models (CHMs) derived from Light Detection And Ranging (LiDAR) data or photographs. However, these algorithms often lead to inaccurate estimates of forest stand characteristics due to the limited information content of raster CHMs. Here we develop a 3D tree delineation method which uses graph cut to delineate trees from the full 3D LiD…
Advanced Hyperspectral Data Analysis Software (AVHYAS) plugin is a python3 based quantum GIS (QGIS) plugin designed to process and analyse hyperspectral (Hx) images. It is developed to guarantee full usage of present and future Hx airborne or spaceborne sensors and provides access to advanced algorithms for Hx data processing. The software is freely available and offers a range of basic and advanced tools such as atmospheric correction (for airborne AVIRISNG image), standard processing tools as well as powerful machine learning and Deep Learning interfaces for Hx data analysis.
Hyperspectral images show similar statistical properties to natural grayscale or color photographic images. However, the classification of hyperspectral images is more challenging because of the very high dimensionality of the pixels and the small number of labeled examples typically available for learning. These peculiarities lead to particular signal processing problems, mainly characterized by indetermination and complex manifolds. The framework of statistical learning has gained popularity in the last decade. New methods have been presented to account for the spatial homogeneity of images,…
Rochester Institute of Technology（罗切斯特理工学院）；ON Semiconductor（安森美半导体）
We investigate applying convolutional neural network (CNN) architecture to facilitate aerial hyperspectral scene understanding and present a new hyperspectral dataset-AeroRIT-that is large enough for CNN training. To date the majority of hyperspectral airborne have been confined to various sub-categories of vegetation and roads and this scene introduces two new categories: buildings and cars. To the best of our knowledge, this is the first comprehensive large-scale hyperspectral scene with nearly seven million pixel annotations for identifying cars, roads, and buildings. We compare the perform…
Feature selection is one of the most important problems in hyperspectral images classification. It consists to choose the most informative bands from the entire set of input datasets and discard the noisy, redundant and irrelevant ones. In this context, we propose a new wrapper method based on normalized mutual information (NMI) and error probability (PE) using support vector machine (SVM) to reduce the dimensionality of the used hyperspectral images and increase the classification efficiency. The experiments have been performed on two challenging hyperspectral benchmarks datasets captured by …
Classification is an important aspect of hyperspectral images processing and application. At present, the researchers mostly use the classic airborne hyperspectral imagery as the benchmark dataset. However, existing datasets suffer from three bottlenecks: (1) low spatial resolution; (2) low labeled pixels proportion; (3) low degree of subclasses distinction. In this paper, a new benchmark dataset named the Wuhan UAV-borne hyperspectral image (WHU-Hi) dataset was built for hyperspectral image classification. The WHU-Hi dataset with a high spectral resolution (nm level) and a very high spatial r…
HyperSAM: A Promptable Foundation Model for Hyperspectral Remote Sensing
Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）；Helmholtz-Zentrum Dresden-Rossendorf（亥姆霍兹德累斯顿罗森多夫研究中心）；University of Iceland（冰岛大学）；Griffith University（格里菲斯大学）
University College London（伦敦大学学院）；Imperial College London（帝国理工学院）；China Academy of Space Technology（中国空间技术研究院）
Hyperspectral images (HSIs) can distinguish materials with high number of spectral bands, which is widely adopted in remote sensing applications and benefits in high accuracy land cover classifications. However, HSIs processing are tangled with the problem of high dimensionality and limited amount of labelled data. To address these challenges, this paper proposes a deep learning architecture using three dimensional convolutional neural networks with spectral partitioning to perform effective feature extraction. We conduct experiments using Indian Pines and Salinas scenes acquired by NASA Airbo…
A PyTorch Library for Hyperspectral Image Models: Technical Report
Siddhartha Academy of Higher Education（悉达多高等教育学院）；Vellore Institute of Technology（韦洛尔理工学院）；Indira Gandhi National Open University（英迪拉·甘地国立开放大学）；Tezpur University（提斯浦尔大学）
Toulouse Hyperspectral Data Set: a benchmark data set to assess semi-supervised spectral representation learning and pixel-wise classification techniques
CNES（法国国家空间研究中心）；Toulouse Mathematics Institute（图卢兹数学研究所）；CNRS（法国国家科学研究中心）；University of Toulouse（图卢兹大学）；ONERA-DOTA（法国航空航天研究院DOTA部门）；Magellium（马哲利姆公司）
针对现有高光谱数据集不适合评估泛化性能的问题，发布大规模Toulouse Hyperspectral Data Set基准，并基于Masked Autoencoder等自监督技术建立像素级分类基线，实现85%准确率。
School of Marine Science and Technology, Northwestern Polytechnical University, China（西北工业大学海洋科学与技术学院）；State Key Laboratory of Rail Traffic Control and Safety, Beijing Jiaotong University, China（北京交通大学轨道交通控制与安全国家重点实验室）
Nowadays, nonnegative matrix factorization (NMF) based methods have been widely applied to blind spectral unmixing. Introducing proper regularizers to NMF is crucial for mathematically constraining the solutions and physically exploiting spectral and spatial properties of images. Generally, properly handcrafting regularizers and solving the associated complex optimization problem are non-trivial tasks. In our work, we propose an NMF based unmixing framework which jointly uses a handcrafting regularizer and a learnt regularizer from data. we plug learnt priors of abundances where the associated…
Cross-domain Hyperspectral Image Classification based on Bi-directional Domain Adaptation
Jockey Club STEM Laboratory of Quantitative Remote Sensing, Department of Geography, the University of Hong Kong（香港大学地理系定量遥感实验室）；School of Information and Electronics, Beijing Institute of Technology（北京理工大学信息与电子学院）；Institute of Forest Resource Information Techniques, Chinese Academy of Forestry（中国林业科学研究院森林资源信息技术研究所）
Computational Geosciences Inc.（计算地球科学公司）；University of British Columbia（不列颠哥伦比亚大学）
The large spatial/frequency scale of hyperspectral and airborne magnetic and gravitational data causes memory issues when using convolutional neural networks for (sub-) surface characterization. Recently developed fully reversible networks can mostly avoid memory limitations by virtue of having a low and fixed memory requirement for storing network states, as opposed to the typical linear memory growth with depth. Fully reversible networks enable the training of deep neural networks that take in entire data volumes, and create semantic segmentations in one go. This approach avoids the need to …
Indian Institute of Technology, Kharagpur（印度理工学院克勒格布尔分校）；University of Pennsylvania（宾夕法尼亚大学）；Columbia University（哥伦比亚大学）；Space Applications Centre, Indian Space Research Organization（印度空间研究组织空间应用中心）
Hyperspectral images of land-cover captured by airborne or satellite-mounted sensors provide a rich source of information about the chemical composition of the materials present in a given place. This makes hyperspectral imaging an important tool for earth sciences, land-cover studies, and military and strategic applications. However, the scarcity of labeled training examples and spatial variability of spectral signature are two of the biggest challenges faced by hyperspectral image classification. In order to address these issues, we aim to develop a framework for material-agnostic informatio…
University of Missouri（密苏里大学）；University of Michigan（密歇根大学）；University of Florida（佛罗里达大学）
In remote sensing, each sensor can provide complementary or reinforcing information. It is valuable to fuse outputs from multiple sensors to boost overall performance. Previous supervised fusion methods often require accurate labels for each pixel in the training data. However, in many remote sensing applications, pixel-level labels are difficult or infeasible to obtain. In addition, outputs from multiple sensors often have different resolution or modalities. For example, rasterized hyperspectral imagery presents data in a pixel grid while airborne Light Detection and Ranging (LiDAR) generates…
BCG-Former: Toward Pareto-Efficient Hyperspectral Image Classification via Band-Contextual Gating
College of Information Science, University of Arizona（亚利桑那大学信息科学学院）；Department of Electrical and Computer Engineering, University of Arizona（亚利桑那大学电气与计算机工程系）
SpecAware: A Spectral-Content Aware Foundation Model for Unifying Multi-Sensor Learning in Hyperspectral Remote Sensing Mapping
Key Laboratory of Spatial-Temporal Big Data Analysis and Application of Natural Resources in Megacities (Ministry of Natural Resources), East China Normal University（空间-时间大数据分析与应用国家级重点实验室（自然资源部），东华师范大学）；School of Geospatial Artificial Intelligence, East China Normal University（地理信息人工智能学院，东华师范大学）；Key Laboratory of Geographic Information Science (Ministry of Education), East China Normal University（地理信息科学重点实验室（教育部），东华师范大学）；School of Geographic Sciences, East China Normal University（地理科学学院，东华师范大学）；Shanghai Municipal Institute of Surveying and Mapping（上海市测绘院）；Institute of Defense Engineering, AMS（工程院，AMS）
AMBER -- Advanced SegFormer for Multi-Band Image Segmentation: an application to Hyperspectral Imaging
University of Naples Federico II（那不勒斯费德里科二世大学）；INFN – Section of Naples（国家核物理研究所那不勒斯分部）；Stazione Zoologica Anton Dohrn（安托万·多尔恩动物站）；INAF - Osservatorio Astronomico di Capodimonte（意大利国家天文研究所卡波迪蒙特天文台）
Hyperspectral Image Classification Based on Faster Residual Multi-branch Spiking Neural Network
Henan University（河南大学）；Henan Key Laboratory of Big Data Analysis and Processing（河南省大数据分析与处理重点实验室）；School of Computer and Information Engineering, Henan University（河南大学计算机与信息工程学院）；Department of Clinical Laboratory, Huaihe Hospital of Henan University（河南大学淮河医院检验科）
Prospects for Mitigating Spectral Variability in Tropical Species Classification Using Self-Supervised Learning
AMAP；Univ Montpellier（蒙彼利埃大学）；CIRAD（法国农业发展研究国际合作中心）；CNRS（法国国家科学研究中心）；INRAE（法国国家农业、食品与环境研究院）；IRD（法国发展研究院）；Technical University of Munich（慕尼黑工业大学）；German Aerospace Center（德国航空航天中心）；Univ. Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；Inria（法国国家信息与自动化研究所）；Grenoble INP（格勒诺布尔理工学院）；LJK（约瑟夫·傅里叶实验室）
