Attention Mechanism Meets with Hybrid Dense Network for Hyperspectral Image Classification
National University of Computer and Emerging Sciences（国立计算机与新兴科学大学）；Innopolis University（因诺波利斯大学）；University of Messina（墨西拿大学）；Jalpaiguri Government Engineering College（贾尔派古里政府工程学院）；Beijing Institute of Technology（北京理工大学）
Muhammad Ahmad, Adil Mehmood Khan, Manuel Mazzara, Salvatore Distefano, Swalpa Kumar Roy, Xin Wu
Convolutional Neural Networks (CNN) are more suitable, indeed. However, fixed kernel sizes make traditional CNN too specific, neither flexible nor conducive to feature learning, thus impacting on the classification accuracy. The convolution of different kernel size networks may overcome this problem by capturing more discriminating and relevant information. In light of this, the proposed solution aims at combining the core idea of 3D and 2D Inception net with the Attention mechanism to boost the HSIC CNN performance in a hybrid scenario. The resulting \textit{attention-fused hybrid network} (AfNet) is based on three attention-fused parallel hybrid sub-nets with different kernels in each block repeatedly using high-level features to enhance the final ground-truth maps. In short, AfNet is able to selectively filter out the discriminative features critical for classification. Several tests on HSI datasets provided competitive results for AfNet compared to state-of-the-art models. The proposed pipeline achieved, indeed, an overall accuracy of 97\% for the Indian Pines, 100\% for Botswana, 99\% for Pavia University, Pavia Center, and Salinas datasets.
PLA Strategic Support Force Information Engineering University（中国人民解放军战略支援部队信息工程大学）；University of Trento（特伦托大学）
Deep learning based methods have seen a massive rise in popularity for hyperspectral image classification over the past few years. However, the success of deep learning is attributed greatly to numerous labeled samples. It is still very challenging to use only a few labeled samples to train deep learning models to reach a high classification accuracy. An active deep-learning framework trained by an end-to-end manner is, therefore, proposed by this paper in order to minimize the hyperspectral image classification costs. First, a deep densely connected convolutional network is considered for hyp…
Hyperspectral Image (HSI) classification based on deep learning has been an attractive area in recent years. However, as a kind of data-driven algorithm, deep learning method usually requires numerous computational resources and high-quality labelled dataset, while the cost of high-performance computing and data annotation is expensive. In this paper, to reduce dependence on massive calculation and labelled samples, we propose a lightweight network architecture (LiteDenseNet) based on DenseNet for Hyperspectral Image Classification. Inspired by GoogLeNet and PeleeNet, we design a 3D two-way de…
When Segmentation Meets Hyperspectral Image: New Paradigm for Hyperspectral Image Classification
Graduate School of Information, Production and Systems, Waseda University（早稻田大学信息、生产与系统研究生院）；Hong Kong Polytechnic University（香港理工大学）；Key Laboratory of Electromagnetic Waves Information, Fudan University（复旦大学电磁波信息重点实验室）
Low-Rank Representations Meets Deep Unfolding: A Generalized and Interpretable Network for Hyperspectral Anomaly Detection
Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）；School of Mathematics and Statistics, Southeast University（东南大学数学学院与统计学院）；College of Resources and Environment, University of Chinese Academy of Sciences（中国科学院大学资源与环境学院）；Univ. Grenoble Alpes, Inria, CNRS, Grenoble INP, LJK（格勒诺布尔阿尔卑斯大学、法国国家信息与自动化研究所、法国国家科学研究中心、格勒诺布尔理工学院、约瑟夫·傅里叶大学联合实验室）
In recent years, Hyperspectral Imaging (HSI) has become a powerful source for reliable data in applications such as remote sensing, agriculture, and biomedicine. However, hyperspectral images are highly data-dense and often benefit from methods to reduce the number of spectral bands while retaining the most useful information for a specific application. We propose a novel band selection method to select a reduced set of wavelengths, obtained from an HSI system in the context of image classification. Our approach consists of two main steps: the first utilizes a filter-based approach to find rel…
CLAReSNet: When Convolution Meets Latent Attention for Hyperspectral Image Classification
Institute of Engineering and Management (IEM), University of Engineering and Management Kolkata, IEM Centre of Excellence for InnovAI, Department of Computer Science & Engineering（工程管理学院（IEM）、工程管理大学科塔卡尔，创新AI研究中心、计算机科学与工程系）；Institute of Engineering and Management (IEM), University of Engineering and Management Kolkata, Department of Computer Science & Engineering（工程管理学院（IEM）、工程管理大学科塔卡尔，计算机科学与工程系）
School of Computer Science, China University of Geosciences（中国地质大学计算机学院）
In the proposed SEHybridSN model, a dense block was used to reuse shallow feature and aimed at better exploiting hierarchical spatial spectral feature. Subsequent depth separable convolutional layers were used to discriminate the spatial information. Further refinement of spatial spectral features was realized by the channel attention method, which were performed behind every 3D convolutional layer and every 2D convolutional layer. Experiment results indicate that our proposed model learn more discriminative spatial spectral features using very few training data. SEHybridSN using only 0.05 and…
School of Life Science and Technology, Xidian University（西安电子科技大学生命科学技术学院）
Face recognition has already been well studied under the visible light and the infrared,in both intra-spectral and cross-spectral cases. However, how to fuse different light bands, i.e., hyperspectral face recognition, is still an open research problem, which has the advantages of richer information retaining and all-weather functionality over single band face recognition. Among the very few works for hyperspectral face recognition, traditional non-deep learning techniques are largely used. Thus, we in this paper bring deep learning into the topic of hyperspectral face recognition, and propose…
imec（比利时微电子研究中心）；Ghent University（根特大学）；Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））
Spatial-Spectral Total Variation (SSTV) can quantify local smoothness of image structures, so it is widely used in hyperspectral image (HSI) processing tasks. Essentially, SSTV assumes a sparse structure of gradient maps calculated along the spatial and spectral directions. In fact, these gradient tensors are not only sparse, but also (approximately) low-rank under FFT, which we have verified by numerical tests and theoretical analysis. Based on this fact, we propose a novel TV regularization to simultaneously characterize the sparsity and low-rank priors of the gradient map (LRSTV). The new r…
Remote Sensing Technology Institute (IMF), German Aerospace Center (DLR)（德国航空航天中心遥感技术研究所）；Univ. Grenoble Alpes（格勒诺布尔大学）；CNRS（法国国家科学研究中心）；Grenoble INP（格勒诺布尔理工学院）；RIKEN Center for Advanced Intelligence Project (AIP)（理化学研究所智能情报研究中心）；Graduate School of Frontier Sciences, the University of Tokyo（东京大学前沿科学研究生院）；Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）；Wuhan University（武汉大学）；INRIA（法国国家信息与自动化研究所）；LJK（格勒诺布尔大学-法国国家科学研究中心-格勒诺布尔理工学院联合实验室）
Hyperspectral imaging, also known as image spectrometry, is a landmark technique in geoscience and remote sensing (RS). In the past decade, enormous efforts have been made to process and analyze these hyperspectral (HS) products mainly by means of seasoned experts. However, with the ever-growing volume of data, the bulk of costs in manpower and material resources poses new challenges on reducing the burden of manual labor and improving efficiency. For this reason, it is, therefore, urgent to develop more intelligent and automatic approaches for various HS RS applications. Machine learning (ML)…
UCLouvain（鲁汶大学）；Instituto de Telecomunicações（电信研究所）；Instituto Superior Técnico（高等技术学院）；Universidade de Lisboa（里斯本大学）
This paper introduces a single-pixel HyperSpectral (HS) imaging framework based on Fourier Transform Interferometry (FTI). By combining a space-time coding of the light illumination with partial interferometric observations of a collimated light beam (observed by a single pixel), our system benefits from (i) reduced measurement rate and light-exposure of the observed object compared to common (Nyquist) FTI imagers, and (ii) high spectral resolution as desirable in, e.g., Fluorescence Spectroscopy (FS). From the principles of compressive sensing with multilevel sampling, our method leverages th…
RIKEN Center for Advanced Intelligence Project (AIP)（理化学研究所先进智能项目中心）；Department of Electronic Engineering, Tsinghua University（清华大学电子工程系）；Paradigm Inc.（第四范式有限公司）；Graduate School of Frontier Sciences, The University of Tokyo（东京大学前沿科学研究生院）；LISMARS, Wuhan University（武汉大学测绘遥感信息工程国家重点实验室）
Non-local low-rank tensor approximation has been developed as a state-of-the-art method for hyperspectral image (HSI) restoration, which includes the tasks of denoising, compressed HSI reconstruction and inpainting. Unfortunately, while its restoration performance benefits from more spectral bands, its runtime also substantially increases. In this paper, we claim that the HSI lies in a global spectral low-rank subspace, and the spectral subspaces of each full band patch group should lie in this global low-rank subspace. This motivates us to propose a unified paradigm combining the spatial and …
Non-local low-rank tensor approximation has been developed as a state-of-the-art method for hyperspectral image (HSI) denoising. Unfortunately, with more spectral bands for HSI, while the running time of these methods significantly increases, their denoising performance benefits little. In this paper, we claim that the HSI underlines a global spectral low-rank subspace, and the spectral subspaces of each full band patch groups should underlie this global low-rank subspace. This motivates us to propose a unified spatial-spectral paradigm for HSI denoising. As the new model is hard to optimize, …
Pontifical Catholic University of Rio de Janeiro（里约热内卢天主教大学）；IBM Research（IBM研究院）；McMaster University（麦克马斯特大学）；National Institute for Space Research (INPE)（巴西国家空间研究所）；Santa Catarina State University (UDESC)（圣卡塔琳娜州立大学）
This work proposes a multi-task fully convolutional architecture for tree species mapping in dense forests from sparse and scarce polygon-level annotations using hyperspectral UAV-borne data. Our model implements a partial loss function that enables dense tree semantic labeling outcomes from non-dense training samples, and a distance regression complementary task that enforces tree crown boundary constraints and substantially improves the model performance. Our multi-task architecture uses a shared backbone network that learns common representations for both tasks and two task-specific decoder…
Rochester Institute of Technology（罗切斯特理工学院）；ON Semiconductor（安森美半导体）
We investigate applying convolutional neural network (CNN) architecture to facilitate aerial hyperspectral scene understanding and present a new hyperspectral dataset-AeroRIT-that is large enough for CNN training. To date the majority of hyperspectral airborne have been confined to various sub-categories of vegetation and roads and this scene introduces two new categories: buildings and cars. To the best of our knowledge, this is the first comprehensive large-scale hyperspectral scene with nearly seven million pixel annotations for identifying cars, roads, and buildings. We compare the perform…
Couger Inc（库格公司）；Thiagarajar college of Engineering（蒂亚加拉贾工程学院）
Hyperspectral Imaging is the acquisition of spectral and spatial information of a particular scene. Capturing such information from a specialized hyperspectral camera remains costly. Reconstructing such information from the RGB image achieves a better solution in both classification and object recognition tasks. This work proposes a novel light weight network with very less number of parameters about 233,059 parameters based on Residual dense model with attention mechanism to obtain this solution. This network uses Coordination Convolutional Block to get the spatial information. The weights fr…
HSSDCT: Factorized Spatial-Spectral Correlation for Hyperspectral Image Fusion
Institute of Data Science, National Cheng Kung University（国立云林大学资料科学研究所）；Institute of Intelligent Systems, National Yang Ming Chiao Tung University（阳明交通大学智能系统研究所）；Department of Electrical Engineering, National Taiwan Normal University（台湾师范大学电子工程系）
Current methods for diagnosing the progression of multiple types of cancer within patients rely on interpreting stained needle biopsies. This process is time-consuming and susceptible to error throughout the paraffinization, Hematoxylin and Eosin (H&E) staining, deparaffinization, and annotation stages. Fourier Transform Infrared (FTIR) imaging has been shown to be a promising alternative to staining for appropriately annotating biopsy cores without the need for deparaffinization or H&E staining with the use of Fourier Transform Infrared (FTIR) images when combined with machine learning to int…
It is extremely important to correctly identify the cultivars of maize seeds in the breeding process of maize. In this paper, the transfer learning as a method of deep learning is adopted to establish a model by combining with the hyperspectral imaging technology. The haploid seeds can be recognized from large amount of diploid maize ones with great accuracy through the model. First, the information of maize seeds on each wave band is collected using the hyperspectral imaging technology, and then the recognition model is built on VGG-19 network, which is pre-trained by large-scale computer vis…
Southwest University（西南大学）；China West Normal University（西华师范大学）
Mining structural priors in data is a widely recognized technique for hyperspectral image (HSI) denoising tasks, whose typical ways include model-based methods and data-based methods. The model-based methods have good generalization ability, while the runtime cannot meet the fast processing requirements of the practical situations due to the large size of an HSI data $ \mathbf{X} \in \mathbb{R}^{MN\times B}$. For the data-based methods, they perform very fast on new test data once they have been trained. However, their generalization ability is always insufficient. In this paper, we propose a …
Due to inadequate energy captured by the hyperspectral camera sensor in poor illumination conditions, low-light hyperspectral images (HSIs) usually suffer from low visibility, spectral distortion, and various noises. A range of HSI restoration methods have been developed, yet their effectiveness in enhancing low-light HSIs is constrained. This work focuses on the low-light HSI enhancement task, which aims to reveal the spatial-spectral information hidden in darkened areas. To facilitate the development of low-light HSI processing, we collect a low-light HSI (LHSI) dataset of both indoor and ou…
University of Minnesota（明尼苏达大学）；University of California-Davis（加州大学戴维斯分校）
Crop production needs to increase in a sustainable manner to meet the growing global demand for food. To identify crop varieties with high yield potential, plant scientists and breeders evaluate the performance of hundreds of lines in multiple locations over several years. To facilitate the process of selecting advanced varieties, an automated framework was developed in this study. A hyperspectral camera was mounted on an unmanned aerial vehicle to collect aerial imagery with high spatial and spectral resolution. Aerial images were captured in two consecutive growing seasons from three experim…
Harbin Institute of Technology（哈尔滨工业大学）；Wuhan University（武汉大学）
High-resolution (HR) hyperspectral face image plays an important role in face related computer vision tasks under uncontrolled conditions, such as low-light environment and spoofing attacks. However, the dense spectral bands of hyperspectral face images come at the cost of limited amount of photons reached a narrow spectral window on average, which greatly reduces the spatial resolution of hyperspectral face images. In this paper, we investigate how to adapt the deep learning techniques to hyperspectral face image super-resolution (HFSR), especially when the training samples are very limited. …
Toulouse Hyperspectral Data Set: a benchmark data set to assess semi-supervised spectral representation learning and pixel-wise classification techniques
CNES（法国国家空间研究中心）；Toulouse Mathematics Institute（图卢兹数学研究所）；CNRS（法国国家科学研究中心）；University of Toulouse（图卢兹大学）；ONERA-DOTA（法国航空航天研究院DOTA部门）；Magellium（马哲利姆公司）
针对现有高光谱数据集不适合评估泛化性能的问题，发布大规模Toulouse Hyperspectral Data Set基准，并基于Masked Autoencoder等自监督技术建立像素级分类基线，实现85%准确率。
University of Waterloo（滑铁卢大学）；Waterloo Artificial Intelligence Institute（滑铁卢人工智能研究所）；Xiamen University（厦门大学）
In this paper, we address the hyperspectral image (HSI) classification task with a generative adversarial network and conditional random field (GAN-CRF) -based framework, which integrates a semi-supervised deep learning and a probabilistic graphical model, and make three contributions. First, we design four types of convolutional and transposed convolutional layers that consider the characteristics of HSIs to help with extracting discriminative features from limited numbers of labeled HSI samples. Second, we construct semi-supervised GANs to alleviate the shortage of training samples by adding…
We present a method to combine markerless motion capture and dense pose feature estimation into a single framework. We demonstrate that dense pose information can help for multiview/single-view motion capture, and multiview motion capture can help the collection of a high-quality dataset for training the dense pose detector. Specifically, we first introduce a novel markerless motion capture method that can take advantage of dense parsing capability provided by the dense pose detector. Thanks to the introduced dense human parsing ability, our method is demonstrated much more efficient, and accu…
X-RAFT: Cross-Modal Non-Rigid Registration of Blue and White Light Neurosurgical Hyperspectral Images
Department of Surgical \& Interventional Engineering, School of Biomedical Engineering \& Imaging Sciences, King's College London, London SE1 7EH Department of Neurosurgery, King's College London Hospital NHS Foundation Trust, London SE5 9RS, UK. Department of Imaging Chemistry \& Biology, School of Biomedical Engineering \& Imaging Sciences, King's College London, London SE1 7EH, UK.
Shenzhen Institute of Future Media Technology（深圳未来媒体技术研究院）；Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；Westlake University（西湖大学）；ETH Zürich（苏黎世联邦理工学院）
Many algorithms have been developed to solve the inverse problem of coded aperture snapshot spectral imaging (CASSI), i.e., recovering the 3D hyperspectral images (HSIs) from a 2D compressive measurement. In recent years, learning-based methods have demonstrated promising performance and dominated the mainstream research direction. However, existing CNN-based methods show limitations in capturing long-range dependencies and non-local self-similarity. Previous Transformer-based methods densely sample tokens, some of which are uninformative, and calculate the multi-head self-attention (MSA) betw…
City University of Hong Kong（香港城市大学）；Singapore Telecommunications Limited（新加坡电信有限公司）；Nanyang Technological University（南洋理工大学）
Various architectures (such as GoogLeNets, ResNets, and DenseNets) have been proposed. However, the existing networks usually suffer from either redundancy of convolutional layers or insufficient utilization of parameters. To handle these challenging issues, we propose Micro-Dense Nets, a novel architecture with global residual learning and local micro-dense aggregations. Specifically, residual learning aims to efficiently retrieve features from different convolutional blocks, while the micro-dense aggregation is able to enhance each block and avoid redundancy of convolutional layers by lessen…
