高光谱成像（HSI）已成为各个领域进行无损质量评估的关键技术，通过空间和光谱数据提供详细信息。尽管其有效，但HSI系统的复杂性和高成本阻碍了其广泛应用。本研究通过探索基于深度学习的RGB（红、绿、蓝）图像高光谱图像重建来解决这些挑战，特别是针对农产品。具体而言，比较了不同的高光谱重建算法，如Hyperspectral Convolutional Neural Network - Dense (HSCNN-D)、High-Resolution Network (HRNET)和Multi-Scale Transformer Plus Plus (MST++)，以评估甘薯的干物质含量。在测试的重建方法中，HRNET表现出优越的性能，实现了最低的平均相对绝对误差（MRAE）0.07、均方根误差（RMSE）0.03，以及最高的峰值信噪比（PSNR）32.28分贝（dB）。使用遗传算法（GA）选择了一些关键特征，并使用可解释人工智能（XAI）解释了它们的重要性。使用RGB、重建和ground truth (GT)数据开发了偏最小二乘回归（PLSR）模型。将这些重建方法的视觉和光谱质量与GT数据进行了比较，并生成了预测图。结果揭示了基于深度学习的高光谱图像重建作为农业和生物应用的成本效益和高效质量评估工具的前景。
英文摘要：
Hyperspectral imaging (HSI) has become a key technology for non-invasive quality evaluation in various fields, offering detailed insights through spatial and spectral data. Despite its efficacy, the complexity and high cost of HSI systems have hindered their widespread adoption. This study addressed these challenges by exploring deep learning-based hyperspectral image reconstruction from RGB (Red, Green, Blue) images, particularly for agricultural products. Specifically, different hyperspectral reconstruction algorithms, such as Hyperspectral Convolutional Neural Network - Dense (HSCNN-D), High-Resolution Network (HRNET), and Multi-Scale Transformer Plus Plus (MST++), were compared to assess the dry matter content of sweet potatoes. Among the tested reconstruction methods, HRNET demonstrated superior performance, achieving the lowest mean relative absolute error (MRAE) of 0.07, root mean square error (RMSE) of 0.03, and the highest peak signal-to-noise ratio (PSNR) of 32.28 decibels (dB). Some key features were selected using the genetic algorithm (GA), and their importance was interpreted using explainable artificial intelligence (XAI). Partial least squares regression (PLSR) models were developed using the RGB, reconstructed, and ground truth (GT) data. The visual and spectra quality of these reconstructed methods was compared with GT data, and predicted maps were generated. The results revealed the prospect of deep learning-based hyperspectral image reconstruction as a cost-effective and efficient quality assessment tool for agricultural and biological applications.
ONERA（法国航空航天研究院）；University Paris Saclay（巴黎-萨克雷大学）；Univ. Bretagne Sud（南布列塔尼大学）；IRISA（法国信息与随机系统研究所）
In recent years, deep learning techniques revolutionized the way remote sensing data are processed. Classification of hyperspectral data is no exception to the rule, but has intrinsic specificities which make application of deep learning less straightforward than with other optical data. This article presents a state of the art of previous machine learning approaches, reviews the various deep learning approaches currently proposed for hyperspectral classification, and identifies the problems and difficulties which arise to implement deep neural networks for this task. In particular, the issues…
Madan Mohan Malaviya University of Technology（马丹·莫汉·马拉维亚技术大学）；Malaviya National Institute of Technology Jaipur（斋浦尔马拉维亚国家理工学院）
In this paper we propose a novel 3D CNN network with localized residual connections for hyperspectral image classification. Our work chalks a comparative study with the existing methods employed for abstracting deeper features and propose a model which incorporates residual features from multiple stages in the network. The proposed architecture processes individual spatiospectral feature rich cubes from hyperspectral images through 3D convolutional layers. The residual connections result in improved performance due to assimilation of both low-level and high-level features. We conduct experimen…
Fraunhofer Heinrich Hertz Institute（弗劳恩霍夫海因里希·赫兹研究所）；Humboldt-University（洪堡大学）
Spectral imaging technologies have rapidly evolved during the past decades. The recent development of single-camera-one-shot techniques for hyperspectral imaging allows multiple spectral bands to be captured simultaneously (3x3, 4x4 or 5x5 mosaic), opening up a wide range of applications. Examples include intraoperative imaging, agricultural field inspection and food quality assessment. To capture images across a wide spectrum range, i.e. to achieve high spectral resolution, the sensor design sacrifices spatial resolution. With increasing mosaic size, this effect becomes increasingly detriment…
A comprehensive review of 3D convolutional neural network-based classification techniques of diseased and defective crops using non-UAV-based hyperspectral images
Hard and soft classification techniques are the conventional ways of image classification on satellite data. These classifiers have number of drawbacks. Firstly, these approaches are inappropriate for mixed pixels. Secondly, these approaches do not consider spatial variability. Kriging based soft classifier (KBSC) is a non-parametric geostatistical method. It exploits the spatial variability of the classes within the image. This letter compares the performance of KBSC with other conventional hard/soft classification techniques. The satellite data used in this study is the Wide Field Sensor (Wi…
Small unmanned aircraft systems (sUAS) have allowed for thousands of aerial images to be captured at a moments notice. The simplicity and relative low cost of flying a sUAS has provided remote sensing practitioners, both commercial and academic, with a viable alternative to traditional remote sensing platforms (airplanes and satellites). This paper is an expanded follow-up study to an initial work. Three radiance-to-reflectance conversion methods were tested to determine the optimal technique to use for converting raw digital count imagery to reflectance maps. The first two methods utilized in…
Current methods for diagnosing the progression of multiple types of cancer within patients rely on interpreting stained needle biopsies. This process is time-consuming and susceptible to error throughout the paraffinization, Hematoxylin and Eosin (H&E) staining, deparaffinization, and annotation stages. Fourier Transform Infrared (FTIR) imaging has been shown to be a promising alternative to staining for appropriately annotating biopsy cores without the need for deparaffinization or H&E staining with the use of Fourier Transform Infrared (FTIR) images when combined with machine learning to int…
Manchester Metropolitan University（曼彻斯特城市大学）；Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）；Brunel University（布鲁内尔大学）
Spectral-spatial based deep learning models have recently proven to be effective in hyperspectral image (HSI) classification for various earth monitoring applications such as land cover classification and agricultural monitoring. However, due to the nature of "black-box" model representation, how to explain and interpret the learning process and the model decision, especially for vegetation classification, remains an open challenge. This study proposes a novel interpretable deep learning model -- a biologically interpretable two-stage deep neural network (BIT-DNN), by incorporating the prior-k…
Recent advances in neural networks have made great progress in the hyperspectral image (HSI) classification. However, the overfitting effect, which is mainly caused by complicated model structure and small training set, remains a major concern. Reducing the complexity of the neural networks could prevent overfitting to some extent, but also declines the networks' ability to express more abstract features. Enlarging the training set is also difficult, for the high expense of acquisition and manual labeling. In this paper, we propose an abundance-based multi-HSI classification method. Firstly, w…
Hyperspectral Image Classification Based on Faster Residual Multi-branch Spiking Neural Network
Henan University（河南大学）；Henan Key Laboratory of Big Data Analysis and Processing（河南省大数据分析与处理重点实验室）；School of Computer and Information Engineering, Henan University（河南大学计算机与信息工程学院）；Department of Clinical Laboratory, Huaihe Hospital of Henan University（河南大学淮河医院检验科）
Silesian University of Technology（西里西亚工业大学）；KP Labs（KP实验室）；European Space Agency（欧洲空间局）
Maintaining farm sustainability through optimizing the agricultural management practices helps build more planet-friendly environment. The emerging satellite missions can acquire multi- and hyperspectral imagery which captures more detailed spectral information concerning the scanned area, hence allows us to benefit from subtle spectral features during the analysis process in agricultural applications. We introduce an approach for extracting 2.5 m cultivated land maps from 10 m Sentinel-2 multispectral image series which benefits from a compact graph convolutional neural network. The experimen…
Hyperspectral Pansharpening: Critical Review, Tools and Future Perspectives
University Federico II（那不勒斯费德里科二世大学）；National Research Council（国家研究委员会）；National Biodiversity Future Center（国家生物多样性未来中心）；INRIA（法国国家信息与自动化研究所）；CNRS（法国国家科学研究中心）；Grenoble INP（格勒诺布尔理工学院）；University of Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；Escuela Politecnica de Caceres（卡塞雷斯理工学院）；University Parthenope（帕尔特诺佩大学）
