Hyperspectral imagery is rich in spatial and spectral information. Using 3D-CNN can simultaneously acquire features of spatial and spectral dimensions to facilitate classification of features, but hyperspectral image information spectral dimensional information redundancy. The use of continuous 3D-CNN will result in a high amount of parameters, and the computational power requirements of the device are high, and the training takes too long. This letter designed the Faster selective kernel mechanism network (FSKNet), FSKNet can balance this problem. It designs 3D-CNN and 2D-CNN conversion modules, using 3D-CNN to complete feature extraction while reducing the dimensionality of spatial and spectrum. However, such a model is not lightweight enough. In the converted 2D-CNN, a selective kernel mechanism is proposed, which allows each neuron to adjust the receptive field size based on the two-way input information scale. Under the Selective kernel mechanism, it mainly includes two components, se module and variable convolution. Se acquires channel dimensional attention and variable convolution to obtain spatial dimension deformation information of ground objects. The model is more accurate, faster, and less computationally intensive. FSKNet achieves high accuracy on the IN, UP, Salinas, and Botswana data sets with very small parameters.
Dual Selective Fusion Transformer Network for Hyperspectral Image Classification
Institute of Artificial Intelligence, School of Computer Science, Wuhan University（武汉大学计算机学院人工智能研究所）；Hubei Luojia Laboratory（湖北珞珈实验室）；Aerospace Information Research Institute, Henan Academy of Sciences（河南省科学院航天信息研究所）
3DSS-Mamba: 3D-Spectral-Spatial Mamba for Hyperspectral Image Classification
Nanjing University of Information Science and Technology（南京信息工程大学）；Institute of Optics and Electronics（光学电子研究所）；China University of Geosciences（中国地质大学）；University of Extremadura（埃斯特雷马杜拉大学）
Imperial College London（帝国理工学院）；University College London（伦敦大学学院）；China Academy of Space Technology（中国空间技术研究院）
Hyperspectral image (HSI) classification has been widely adopted in applications involving remote sensing imagery analysis which require high classification accuracy and real-time processing speed. Methods based on Convolutional neural networks (CNNs) have been proven to achieve state-of-the-art accuracy in classifying HSIs. However, CNN models are often too computationally intensive to achieve real-time response due to the high dimensional nature of HSI, compared to traditional methods such as Support Vector Machines (SVMs). Besides, previous CNN models used in HSI are not specially designed …
Hyperspectral Image Classification Based on Faster Residual Multi-branch Spiking Neural Network
Henan University（河南大学）；Henan Key Laboratory of Big Data Analysis and Processing（河南省大数据分析与处理重点实验室）；School of Computer and Information Engineering, Henan University（河南大学计算机与信息工程学院）；Department of Clinical Laboratory, Huaihe Hospital of Henan University（河南大学淮河医院检验科）
EnergyFormer: Energy Attention with Fourier Embedding for Hyperspectral Image Classification
Department of Computer Science, National University of Computer and Emerging Sciences（计算机科学系，国立计算机与新兴科学大学）；Institute of Software Development and Engineering, Innopolis University（软件开发与工程研究所，因诺波利斯大学）；Dipartimento di Matematica e Informatica—MIFT, University of Messina（数学与信息学系—MIFT，墨西拿大学）
National University of Computer and Emerging Sciences（国立计算机与新兴科学大学）；Innopolis University（因诺波利斯大学）；University of Messina（墨西拿大学）；Jalpaiguri Government Engineering College（贾尔派古里政府工程学院）；Beijing Institute of Technology（北京理工大学）
Convolutional Neural Networks (CNN) are more suitable, indeed. However, fixed kernel sizes make traditional CNN too specific, neither flexible nor conducive to feature learning, thus impacting on the classification accuracy. The convolution of different kernel size networks may overcome this problem by capturing more discriminating and relevant information. In light of this, the proposed solution aims at combining the core idea of 3D and 2D Inception net with the Attention mechanism to boost the HSIC CNN performance in a hybrid scenario. The resulting \textit{attention-fused hybrid network} (A…
Nanjing University（南京大学）；Beijing Institute of Technology（北京理工大学）
Deep learning methods have been successfully applied to hyperspectral image (HSI) classification with remarkable performance. Because of limited labelled HSI data, earlier studies primarily adopted a patch-based classification framework, which divides images into overlapping patches for training and testing. However, this approach results in redundant computations and possible information leakage. In this study, we propose an objective evaluation-based high-efficiency learning framework for tiny HSI classification. This framework comprises two main parts: (i) a leakage-free balanced sampling s…
University of Tabriz（大不里士大学）；Ryerson University（瑞尔森大学）
Hyperspectral images provide abundant spatial and spectral information that is very valuable for material detection in diverse areas of practical science. The high-dimensions of data lead to many processing challenges that can be addressed via existent spatial and spectral redundancies. In this paper, a sparse modeling framework is proposed for hyperspectral image classification. Spectral blocks are introduced to be used along with spatial groups to jointly exploit spectral and spatial redundancies. To reduce the computational complexity of sparse modeling, spectral blocks are used to break th…
IGroupSS-Mamba: Interval Group Spatial-Spectral Mamba for Hyperspectral Image Classification
Nanjing University of Information Science and Technology（南京信息工程大学）；Institute of Optics and Electronics（光学电子研究所）；China University of Geosciences（中国地质大学）；University of Extremadura（埃斯特雷马杜拉大学）
S$^2$Mamba: A Spatial-spectral State Space Model for Hyperspectral Image Classification
School of Artificial Intelligence, Xidian University（西安电子科技大学人工智能学院）；MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University（上海交通大学人工智能教育部重点实验室、人工智能研究院）
ERX: A Fast Real-Time Anomaly Detection Algorithm for Hyperspectral Line Scanning
School of Aerospace, Mechanical, and Mechatronic Engineering, The University of Sydney（悉尼大学航空航天、机械与机电工程学院）；School of Environmental and Rural Science, The University of New England（新英格兰大学环境与农村科学学院）；Defence Science and Technology Group（国防科学技术集团）；Australian Research Council (ARC) Training Centre for CubeSats, UAVs & Their Applications (CUAVA)（澳大利亚研究委员会立方体卫星、无人机及其应用培训中心）
HS-Mamba: Full-Field Interaction Multi-Groups Mamba for Hyperspectral Image Classification
College of Mathematics and Informatics, South China Agricultural University（华南农业大学数学与信息学院）；Key Laboratory of Smart Agricultural Technology in Tropical South China, Ministry of Agriculture and Rural Affairs（热带南方智能农业技术重点实验室，农业部）；School of Chemistry and Chemical Engineering, South China University of Technology（华南理工大学化学与化工学院）
Transformer-Driven Active Transfer Learning for Cross-Hyperspectral Image Classification
SDAIA-KFUPM, Joint Research Center for Artificial Intelligence (JRCAI), King Fahd University of Petroleum and Minerals（SDAIA-KFUPM、联合人工智能研究中心（JRCAI）、国王法赫德石油大学）；Engineering Department, University of Sannio（工程系，萨尼奥大学）；Institute of Software Development and Engineering, Innopolis University（软件开发与工程研究所，因诺波利斯大学）；Dipartimento di Matematica e Informatica-MIFT, University of Messina（数学与信息学系-MIFT，梅斯辛纳大学）；School of Computer Science, University of Hull（计算机科学学院，霍尔大学）
SpectralMamba: Efficient Mamba for Hyperspectral Image Classification
Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）；School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences（中国科学院大学电子电气与通信工程学院）；School of Mathematics and Statistics, Southeast University（东南大学数学学院）；Univ. Grenoble Alpes, Inria, CNRS, Grenoble INP, LJK（格勒诺布尔阿尔卑斯大学、法国国家信息与自动化研究所、法国国家科学研究中心、格勒诺布尔理工学院、约瑟夫·傅里叶大学）
Exploring Multi-Timestep Multi-Stage Diffusion Features for Hyperspectral Image Classification
School of Information Science and Technology, Fudan University（复旦大学信息科学与技术学院）；Academy for Engineering and Technology, Fudan University（复旦大学工程与技术学院）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）
SSUMamba: Spatial-Spectral Selective State Space Model for Hyperspectral Image Denoising
School of Computer Science and Engineering, Nanjing University of Science and Technology（南京理工大学计算机科学与工程学院）；School of Information and Communication Technology, Griffith University（格里菲斯大学信息与通信技术学院）
该研究提出 SSUMamba，通过六种空间-光谱连续扫描顺序、双向 SSM 与 3D 卷积高效建模高光谱图像的长程及局部依赖，在降低每批内存消耗的同时提升去噪性能。
In recent times, CNNs have made significant contributions to applications in image generation, super-resolution and style transfer. In this paper, we build upon the work of Howard and Gugger, He et al. and Misra, D. and propose a CNN architecture that accurately reconstructs hyperspectral images from their RGB counterparts. We also propose a much shallower version of our best model with a 10% relative memory footprint and 3x faster inference, thus enabling real-time video applications while still experiencing only about a 0.5% decrease in performance.
Spectral imaging enables spatially-resolved identification of materials in remote sensing, biomedicine, and astronomy. However, acquisition times require balancing spectral and spatial resolution with signal-to-noise. Hyperspectral imaging provides superior material specificity, while multispectral images are faster to collect at greater fidelity. We propose an approach for fusing hyperspectral and multispectral images to provide high-quality hyperspectral output. The proposed optimization leverages the least absolute shrinkage and selection operator (LASSO) to perform variable selection and r…
Spectral Dynamic Attention Network for Hyperspectral Image Super-Resolution
Sanya Oceanographic Institution, Ocean University of China（南海海洋研究所，中国海洋大学）；State Key Laboratory of Physical Oceanography, Ocean University of China（物理海洋学国家重点实验室，中国海洋大学）；Department of Electrical and Computer Engineering, Mississippi State University（密苏里州立大学电气与计算机工程系）
THAT: Token-wise High-frequency Augmentation Transformer for Hyperspectral Pansharpening
JPMorgan Chase（摩根大通公司）；University of Missouri-Kansas City（密苏里大学-堪萨斯城分校）；Ming Hsieh Department of Electrical and Computer Engineering（明希赫电气与计算机工程系）；University of Southern California（南加州大学）；Robinson Research Institute（罗宾逊研究学院）；KTH Royal Institute of Technology（皇家理工学院）；NEC Laboratories America（NEC美国实验室）；University of Virginia（弗吉尼亚大学）
Compression of hyperspectral images onboard of spacecrafts is a tradeoff between the limited computational resources and the ever-growing spatial and spectral resolution of the optical instruments. As such, it requires low-complexity algorithms with good rate-distortion performance and high throughput. In recent years, the Consultative Committee for Space Data Systems (CCSDS) has focused on lossless and near-lossless compression approaches based on predictive coding, resulting in the recently published CCSDS 123.0-B-2 recommended standard. While the in-loop reconstruction of quantized predicti…
HDMba: Hyperspectral Remote Sensing Imagery Dehazing with State Space Model
College of Oceanography and Space Informatics, China University of Petroleum (East China)（中国石油大学（华东）海洋与空间信息学院）；National Subsea Centre, Robert Gordon University（罗伯特戈登大学国家海底中心）；Helmholtz Institute Freiberg for Resource Technology, Helmholtz-Zentrum Dresden-Rossendorf（亥姆霍兹德累斯顿罗森多夫研究中心亥姆霍兹弗赖贝格资源技术研究所）
Politecnico di Milano（米兰理工大学）；Harvard University（哈佛大学）
Image quality plays a big role in CNN-based image classification performance. Fine-tuning the network with distorted samples may be too costly for large networks. To solve this issue, we propose a transfer learning approach optimized to keep into account that in each layer of a CNN some filters are more susceptible to image distortion than others. Our method identifies the most susceptible filters and applies retraining only to the filters that show the highest activation maps distance between clean and distorted images. Filters are ranked using the Borda count election method and then only th…
Lightweight Interpretable RGB-Guided Hyperspectral Super-Resolution under Real Cross-resolution Misalignment
Univ. Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；CNRS Grenoble INP（法国国家科学研究中心格勒诺布尔国立理工学院）；GIPSA-Lab（格勒诺布尔图像、信号、声学与自动化实验室）；Institut Universitaire de France (IUF)（法国大学研究院）
Imperial College London（伦敦帝国学院）；University College London（伦敦大学学院）
Purpose: Intra-operative measurement of tissue oxygen saturation (StO2) is important in the detection of ischemia, monitoring perfusion and identifying disease. Hyperspectral imaging (HSI) measures the optical reflectance spectrum of the tissue and uses this information to quantify its composition, including StO2. However, real-time monitoring is difficult due to the capture rate and data processing time. Methods: An endoscopic system based on a multi-fiber probe was previously developed to sparsely capture HSI data (sHSI). These were combined with RGB images, via a deep neural network, to gen…
Here we apply hyperspectral bright field imaging to collect computed tomographic images with excellent energy resolution (800 eV), applying it for the first time to map the distribution of stain in a fixed biological sample through its characteristic K-edge. Conventionally, because the photons detected at each pixel are distributed across as many as 200 energy channels, energy-selective images are characterised by low count-rates and poor signal-to-noise ratio. This means high X-ray exposures, long scan times and high doses are required to image unique spectral markers. Here, we achieve high q…
Accurate land cover segmentation of spectral images is challenging and has drawn widespread attention in remote sensing due to its inherent complexity. Although significant efforts have been made for developing a variety of methods, most of them rely on supervised strategies. Subspace clustering methods, such as Sparse Subspace Clustering (SSC), have become a popular tool for unsupervised learning due to their high performance. However, the computational complexity of SSC methods prevents their use on large spectral remotely sensed datasets. Furthermore, since SSC ignores the spatial informati…
The hyperspectral image (HSI) unmixing task is essentially an inverse problem, which is commonly solved by optimization algorithms under a predefined (non-)linear mixture model. Although these optimization algorithms show impressive performance, they are very computational demanding as they often rely on an iterative updating scheme. Recently, the rise of neural networks has inspired lots of learning based algorithms in unmixing literature. However, most of them lack of interpretability and require a large training dataset. One natural question then arises: can one leverage the model based alg…
Carl Zeiss AG（卡尔蔡司股份公司）；University of Padua（帕多瓦大学）；University of Applied Sciences Western Switzerland (HES-SO Valais)（瑞士西部应用科学大学（瓦莱州高等专业学院））；Carl Zeiss Meditec AG（卡尔蔡司医疗科技股份公司）；IMEC（比利时微电子研究中心）；University Hospitals Leuven（鲁汶大学医院）；KU Leuven（鲁汶大学）
Surgery for gliomas (intrinsic brain tumors), especially when low-grade, is challenging due to the infiltrative nature of the lesion. Currently, no real-time, intra-operative, label-free and wide-field tool is available to assist and guide the surgeon to find the relevant demarcations for these tumors. While marker-based methods exist for the high-grade glioma case, there is no convenient solution available for the low-grade case; thus, marker-free optical techniques represent an attractive option. Although RGB imaging is a standard tool in surgical microscopes, it does not contain sufficient …
Brillouin imaging relies on the reliable extraction of subtle spectral information from hyperspectral datasets. To date, the mainstream practice has been using line fitting of spectral features to retrieve the average peak shift and linewidth parameters. Good results, however, depend heavily on sufficient SNR and may not be applicable in complex samples that consist of spectral mixtures. In this work, we thus propose the use of various multivariate algorithms that can be used to perform supervised or unsupervised analysis of the hyperspectral data, with which we explore advanced image analysis…
Signal extrapolation tasks arise in miscellaneous manners in the field of image and video signal processing. But, due to the widespread use of low-power and mobile devices, the computational complexity of an algorithm plays a crucial role in selecting an algorithm for a given problem. Within the scope of this contribution, we introduce the complex-valued Frequency Selective Extrapolation for fast image and video signal extrapolation. This algorithm iteratively generates a generic complex-valued model of the signal to be extrapolated as weighted superposition of Fourier basis functions. We furt…
Many applications in image processing require resampling of arbitrarily located samples onto regular grid positions. This is important in frame-rate up-conversion, super-resolution, and image warping among others. A state-of-the-art high quality model-based resampling technique is frequency-selective mesh-to-grid resampling which requires pre-estimation of key points. In this paper, we propose a new key point agnostic frequency-selective mesh-to-grid resampling that does not depend on pre-estimated key points. Hence, the number of data points that are included is reduced drastically and the ru…
Less is More: Selective Reduction of CT Data for Self-Supervised Pre-Training of Deep Learning Models with Contrastive Learning Improves Downstream Classification Performance
Ulm University（乌尔姆大学）；Ulm University Medical Center（乌尔姆大学医学中心）
