Recent advances in neural networks have made great progress in the hyperspectral image (HSI) classification. However, the overfitting effect, which is mainly caused by complicated model structure and small training set, remains a major concern. Reducing the complexity of the neural networks could prevent overfitting to some extent, but also declines the networks' ability to express more abstract features. Enlarging the training set is also difficult, for the high expense of acquisition and manual labeling. In this paper, we propose an abundance-based multi-HSI classification method. Firstly, we convert every HSI from the spectral domain to the abundance domain by a dataset-specific autoencoder. Secondly, the abundance representations from multiple HSIs are collected to form an enlarged dataset. Lastly, we train an abundance-based classifier and employ the classifier to predict over all the involved HSI datasets. Different from the spectra that are usually highly mixed, the abundance features are more representative in reduced dimension with less noise. This benefits the proposed method to employ simple classifiers and enlarged training data, and to expect less overfitting issues. The effectiveness of the proposed method is verified by the ablation study and the comparative experiments.
Institute of Theoretical and Applied Informatics, Polish Academy of Sciences（波兰科学院理论与应用信息学研究所）；Silesian University of Technology（西里西亚工业大学）；Sapientia Hungarian University of Transylvania（特兰西瓦尼亚萨皮恩扎匈牙利大学）
Neural networks, in particular autoencoders, are one of the most promising solutions for unmixing hyperspectral data, i.e. reconstructing the spectra of observed substances (endmembers) and their relative mixing fractions (abundances), which is needed for effective hyperspectral analysis and classification. However, as we show in this paper, the training of autoencoders for unmixing is highly dependent on weights initialisation; some sets of weights lead to degenerate or low-performance solutions, introducing negative bias in the expected performance. In this work, we experimentally investigat…
In this paper, we introduce a new algorithm based on archetypal analysis for blind hyperspectral unmixing, assuming linear mixing of endmembers. Archetypal analysis is a natural formulation for this task. This method does not require the presence of pure pixels (i.e., pixels containing a single material) but instead represents endmembers as convex combinations of a few pixels present in the original hyperspectral image. Our approach leverages an entropic gradient descent strategy, which (i) provides better solutions for hyperspectral unmixing than traditional archetypal analysis algorithms, an…
Southwest Jiaotong University（西南交通大学）；Mississippi State University（密西西比州立大学）；University of New South Wales（新南威尔士大学）；University of Extremadura（埃斯特雷马杜拉大学）
Hyperspectral unmixing has been an important technique that estimates a set of endmembers and their corresponding abundances from a hyperspectral image (HSI). Nonnegative matrix factorization (NMF) plays an increasingly significant role in solving this problem. In this article, we present a comprehensive survey of the NMF-based methods proposed for hyperspectral unmixing. Taking the NMF model as a baseline, we show how to improve NMF by utilizing the main properties of HSIs (e.g., spectral, spatial, and structural information). We categorize three important development directions including con…
Although many spectral unmixing models have been developed to address spectral variability caused by variable incident illuminations, the mechanism of the spectral variability is still unclear. This paper proposes an unmixing model, named illumination invariant spectral unmixing (IISU). IISU makes the first attempt to use the radiance hyperspectral data and a LiDAR-derived digital surface model (DSM) in order to physically explain variable illuminations and shadows in the unmixing framework. Incident angles, sky factors, visibility from the sun derived from the LiDAR-derived DSM support the ex…
Transformer for Multitemporal Hyperspectral Image Unmixing
College of Computer Science, Nankai University（南开大学计算机学院）；Haihe Lab（海河实验室）；School of Computer Science and Technology, Tiangong University（天津工业大学计算机科学与技术学院）；School of Astronautics, Beihang University（北京航空航天大学宇航学院）
University of Florida（佛罗里达大学）；UiT - The Arctic University of Norway（UiT-挪威北极大学）
We develop a new neural network based independent component analysis (ICA) method by directly minimizing the dependence amongst all extracted components. Using the matrix-based R{é}nyi's $α$-order entropy functional, our network can be directly optimized by stochastic gradient descent (SGD), without any variational approximation or adversarial training. As a solid application, we evaluate our ICA in the problem of hyperspectral unmixing (HU) and refute a statement that "\emph{ICA does not play a role in unmixing hyperspectral data}", which was initially suggested by \cite{nascimento2005does}. …
In the remote sensing context spectral unmixing is a technique to decompose a mixed pixel into two fundamental representatives: endmembers and abundances. In this paper, a novel architecture is proposed to perform blind unmixing on hyperspectral images. The proposed architecture consists of convolutional layers followed by an autoencoder. The encoder transforms the feature space produced through convolutional layers to a latent space representation. Then, from these latent characteristics the decoder reconstructs the roll-out image of the monochrome image which is at the input of the architect…
German Aerospace Center (DLR)（德国航空航天中心）；Univ. Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）；The University of Tokyo（东京大学）；RIKEN Center for Advanced Intelligence Project (AIP)（理化学研究所先进智能项目中心）
Over the past decades, enormous efforts have been made to improve the performance of linear or nonlinear mixing models for hyperspectral unmixing, yet their ability to simultaneously generalize various spectral variabilities and extract physically meaningful endmembers still remains limited due to the poor ability in data fitting and reconstruction and the sensitivity to various spectral variabilities. Inspired by the powerful learning ability of deep learning, we attempt to develop a general deep learning approach for hyperspectral unmixing, by fully considering the properties of endmembers e…
Mixed pixels are presented in hyperspectral images due to low spatial resolution of hyperspectral sensors. Spectral unmixing decomposes mixed pixels spectra into endmembers spectra and abundance fractions. In this paper using of robust statistics-based nonnegative matrix factorization (RNMF) for spectral unmixing of hyperspectral data is investigated. RNMF uses a robust cost function and iterative updating procedure, so is not sensitive to outliers. This method has been applied to simulated data using USGS spectral library, AVIRIS and ROSIS datasets. Unmixing results are compared to traditiona…
University of Florida（佛罗里达大学）；University of Missouri（密苏里大学）
A semi-supervised Partial Membership Latent Dirichlet Allocation approach is developed for hyperspectral unmixing and endmember estimation while accounting for spectral variability and spatial information. Partial Membership Latent Dirichlet Allocation is an effective approach for spectral unmixing while representing spectral variability and leveraging spatial information. In this work, we extend Partial Membership Latent Dirichlet Allocation to incorporate any available (imprecise) label information to help guide unmixing. Experimental results on two hyperspectral datasets show that the propo…
Department of Engineering Cybernetics, Norwegian University of Science and Technology（工程 cybernetics 部，挪威科学与技术大学）；Department of Biology, Norwegian University of Science and Technology（生物学部，挪威科学与技术大学）；Department of Automatic Control and Systems Engineering, University Politehnica Bucharest（自动控制与系统工程部，布加勒斯特理工大学）
Silesian University of Technology（西里西亚工业大学）；KP Labs（KP实验室）；European Space Agency（欧洲空间局）
Hyperspectral unmixing remains one of the most challenging tasks in the analysis of such data. Deep learning has been blooming in the field and proved to outperform other classic unmixing techniques, and can be effectively deployed onboard Earth observation satellites equipped with hyperspectral imagers. In this letter, we follow this research pathway and propose a multi-branch convolutional neural network that benefits from fusing spectral, spatial, and spectral-spatial features in the unmixing process. The results of our experiments, backed up with the ablation study, revealed that our techn…
Jalpaiguri Engineering College（贾尔派古里工程学院）；University of Antwerp（安特卫普大学）；Helmholtz-Zentrum Dresden-Rossendorf（亥姆霍兹德累斯顿罗森多夫研究中心）；Helmholtz Institute Freiberg for Resource Technology（亥姆霍兹弗赖贝格资源技术研究所）
Currently, this paper is under review in IEEE. Transformers have intrigued the vision research community with their state-of-the-art performance in natural language processing. With their superior performance, transformers have found their way in the field of hyperspectral image classification and achieved promising results. In this article, we harness the power of transformers to conquer the task of hyperspectral unmixing and propose a novel deep unmixing model with transformers. We aim to utilize the ability of transformers to better capture the global feature dependencies in order to enhanc…
Qom University of Technology（库姆理工大学）；Freie Universität Berlin（柏林自由大学）
Spectral unmixing (SU) of hyperspectral images (HSIs) is one of the important areas in remote sensing (RS) that needs to be carefully addressed in different RS applications. Despite the high spectral resolution of the hyperspectral data, the relatively low spatial resolution of the sensors may lead to mixture of different pure materials within the image pixels. In this case, the spectrum of a given pixel recorded by the sensor can be a combination of multiple spectra each belonging to a unique material in that pixel. Spectral unmixing is then used as a technique to extract the spectral charact…
School of Computer Science, Wuhan University（武汉大学计算机学院）
Characterized by tremendous spectral information, hyperspectral image is able to detect subtle changes and discriminate various change classes for change detection. The recent research works dominated by hyperspectral binary change detection, however, cannot provide fine change classes information. And most methods incorporating spectral unmixing for hyperspectral multiclass change detection (HMCD), yet suffer from the neglection of temporal correlation and error accumulation. In this study, we proposed an unsupervised Binary Change Guided hyperspectral multiclass change detection Network (BCG…
Northwestern Polytechnical University（西北工业大学）；Mississippi State University（密西西比州立大学）
Spectral unmixing (SU) expresses the mixed pixels existed in hyperspectral images as the product of endmember and abundance, which has been widely used in hyperspectral imagery analysis. However, the influence of light, acquisition conditions and the inherent properties of materials, results in that the identified endmembers can vary spectrally within a given image (construed as spectral variability). To address this issue, recent methods usually use a priori obtained spectral library to represent multiple characteristic spectra of the same object, but few of them extracted the spectral variab…
Hyperspectral Image Generation with Unmixing Guided Diffusion Model
School of Statistics and Data Science, KLMDASR, LEBPS, and LPMC, Nankai University（统计与数据科学学院，KLMDASR，LEBPS和LPMC，南开大学）；Image Processing Center, School of Astronautics, Beihang University（图像处理中心，航天学院，北航大学）
A Dual Symmetric Gauss-Seidel Alternating Direction Method of Multipliers for Hyperspectral Sparse Unmixing
School of Information Science and technology, and the Provincial Key Lab of Information Coding and Trans- mission, Southwest Jiaotong University（信息科学与技术学院，信息编码与传输省重点实验室，西南交通大学）；School of Mathematics, Southwest Jiaotong University（数学学院，西南交通大学）；School of Computer and Computing Science, Zhejiang University City College（计算机与计算科学学院，浙江大学城市学院）
An efficient spatial regularization method using superpixel segmentation and graph Laplacian regularization is proposed for sparse hyperspectral unmixing method. Since it is likely to find spectrally similar pixels in a homogeneous region, we use a superpixel segmentation algorithm to extract the homogeneous regions by considering the image boundaries. We first extract the homogeneous regions, which are called superpixels, then a weighted graph in each superpixel is constructed by selecting $K$-nearest pixels in each superpixel. Each node in the graph represents the spectrum of a pixel and edg…
GIPSA-lab/DIS（GIPSA实验室/DIS）；Institut Universitaire de France (IUF)（法国大学研究院）
In this paper, we consider the problem of unmixing a time series of hyperspectral images. We propose a dynamical model based on linear mixing processes at each time instant. The spectral signatures and fractional abundances of the pure materials in the scene are seen as latent variables, and assumed to follow a general dynamical structure. Based on a simplified version of this model, we derive an efficient spectral unmixing algorithm to estimate the latent variables by performing alternating minimizations. The performance of the proposed approach is demonstrated on synthetic and real multitemp…
Technion Israel Institute of Technology（以色列理工学院）；Bar-Ilan University（巴伊兰大学）
Training a deep neural network for classification constitutes a major problem in remote sensing due to the lack of adequate field data. Acquiring high-resolution ground truth (GT) by human interpretation is both cost-ineffective and inconsistent. We propose, instead, to utilize high-resolution, hyperspectral images for solving this problem, by unmixing these images to obtain reliable GT for training a deep network. Specifically, we simulate GT from high-resolution, hyperspectral FENIX images, and use it for training a convolutional neural network (CNN) for pixel-based classification. We show h…
School of Marine Science and Technology, Northwestern Polytechnical University, China（西北工业大学海洋科学与技术学院）；State Key Laboratory of Rail Traffic Control and Safety, Beijing Jiaotong University, China（北京交通大学轨道交通控制与安全国家重点实验室）
Nowadays, nonnegative matrix factorization (NMF) based methods have been widely applied to blind spectral unmixing. Introducing proper regularizers to NMF is crucial for mathematically constraining the solutions and physically exploiting spectral and spatial properties of images. Generally, properly handcrafting regularizers and solving the associated complex optimization problem are non-trivial tasks. In our work, we propose an NMF based unmixing framework which jointly uses a handcrafting regularizer and a learnt regularizer from data. we plug learnt priors of abundances where the associated…
This paper introduces a graph Laplacian regularization in the hyperspectral unmixing formulation. The proposed regularization relies upon the construction of a graph representation of the hyperspectral image. Each node in the graph represents a pixel's spectrum, and edges connect spectrally and spatially similar pixels. The proposed graph framework promotes smoothness in the estimated abundance maps and collaborative estimation between homogeneous areas of the image. The resulting convex optimization problem is solved using the Alternating Direction Method of Multipliers (ADMM). A special atte…
Spectral unmixing is an important tool in hyperspectral data analysis for estimating endmembers and abundance fractions in a mixed pixel. This paper examines the applicability of a recently developed algorithm called graph regularized nonnegative matrix factorization (GNMF) for this aim. The proposed approach exploits the intrinsic geometrical structure of the data besides considering positivity and full additivity constraints. Simulated data based on the measured spectral signatures, is used for evaluating the proposed algorithm. Results in terms of abundance angle distance (AAD) and spectral…
Hyperspectral remote sensing is a prominent research topic in data processing. Most of the spectral unmixing algorithms are developed by adopting the linear mixing models. Nonnegative matrix factorization (NMF) and its developments are used widely for estimation of signatures and fractional abundances in the SU problem. Sparsity constraints was added to NMF, and was regularized by $ L_ {q} $ norm. In this paper, at first hyperspectral images are clustered by fuzzy c- means method, and then a new algorithm based on sparsity constrained distributed optimization is used for spectral unmixing. In …
Hyperspectral images contain mixed pixels due to low spatial resolution of hyperspectral sensors. Mixed pixels are pixels containing more than one distinct material called endmembers. The presence percentages of endmembers in mixed pixels are called abundance fractions. Spectral unmixing problem refers to decomposing these pixels into a set of endmembers and abundance fractions. Due to nonnegativity constraint on abundance fractions, nonnegative matrix factorization methods (NMF) have been widely used for solving spectral unmixing problem. In this paper we have used graph regularized (GNMF) me…
One of the challenges in hyperspectral data analysis is the presence of mixed pixels. Mixed pixels are the result of low spatial resolution of hyperspectral sensors. Spectral unmixing methods decompose a mixed pixel into a set of endmembers and abundance fractions. Due to nonnegativity constraint on abundance fraction values, NMF based methods are well suited to this problem. In this paper multilayer NMF has been used to improve the results of NMF methods for spectral unmixing of hyperspectral data under the linear mixing framework. Sparseness constraint on both spectral signatures and abundan…
Qom University of Technology（库姆理工大学）；Shiraz University（设拉子大学）
Spectral unmixing (SU) is a technique to characterize mixed pixels in hyperspectral images measured by remote sensors. Most of the spectral unmixing algorithms are developed using the linear mixing models. To estimate endmembers and fractional abundance matrices in a blind problem, nonnegative matrix factorization (NMF) and its developments are widely used in the SU problem. One of the constraints which was added to NMF is sparsity, that was regularized by Lq norm. In this paper, a new algorithm based on distributed optimization is suggested for spectral unmixing. In the proposed algorithm, a …
Arizona State University（亚利桑那州立大学）；Northern Arizona University（北亚利桑那大学）
Hyperspectral unmixing is an important remote sensing task with applications including material identification and analysis. Characteristic spectral features make many pure materials identifiable from their visible-to-infrared spectra, but quantifying their presence within a mixture is a challenging task due to nonlinearities and factors of variation. In this paper, spectral variation is considered from a physics-based approach and incorporated into an end-to-end spectral unmixing algorithm via differentiable programming. The dispersion model is introduced to simulate realistic spectral variat…
Universite Bourgogne Franche-Comté（勃艮第-弗朗什-孔泰大学）；Le Centre de Recherche et de Restauration des Musées de France（法国博物馆研究与修复中心）；Centre National de la Recherche Scientifique（法国国家科学研究中心）
In the past few decades, there has been intensive research concerning the Unmixing of hyperspectral images. Some methods such as NMF, VCA, and N-FINDR have become standards since they show robustness in dealing with the unmixing of hyperspectral images. However, the research concerning the unmixing of multispectral images is relatively scarce. Thus, we extend some unmixing methods to the multispectral images. In this paper, we have created two simulated multispectral datasets from two hyperspectral datasets whose ground truths are given. Then we apply the unmixing methods (VCA, NMF, N-FINDR) t…
Spectral unmixing (SU) is a data processing problem in hyperspectral remote sensing. The significant challenge in the SU problem is how to identify endmembers and their weights, accurately. For estimation of signature and fractional abundance matrices in a blind problem, nonnegative matrix factorization (NMF) and its developments are used widely in the SU problem. One of the constraints which was added to NMF is sparsity constraint that was regularized by L 1/2 norm. In this paper, a new algorithm based on distributed optimization has been used for spectral unmixing. In the proposed algorithm,…
Federal University of Santa Catarina（圣卡塔琳娜联邦大学）；Université de Nice Sophia-Antipolis（尼斯索菲亚-安蒂波利斯大学）；CNRS（法国国家科学研究中心）；University of Toulouse（图卢兹大学）；IRIT-ENSEEIHT
The profusion of spectral bands generated by the acquisition process of hyperspectral images generally leads to high computational costs. Such difficulties arise in particular with nonlinear unmixing methods, which are naturally more complex than linear ones. This complexity, associated with the high redundancy of information within the complete set of bands, make the search of band selection algorithms relevant. With this work, we propose a band selection strategy in reproducing kernel Hilbert spaces that allows to drastically reduce the processing time required by nonlinear unmixing techniqu…
University of Toulouse（图卢兹大学）；IRIT/INP-ENSEEIHT（图卢兹计算机科学研究所/国立高等电子技术、电子、计算机、水力学与电信学院）
Hyperspectral unmixing is aimed at identifying the reference spectral signatures composing an hyperspectral image and their relative abundance fractions in each pixel. In practice, the identified signatures may vary spectrally from an image to another due to varying acquisition conditions, thus inducing possibly significant estimation errors. Against this background, hyperspectral unmixing of several images acquired over the same area is of considerable interest. Indeed, such an analysis enables the endmembers of the scene to be tracked and the corresponding endmember variability to be charact…
Key Laboratory of Digital Earth Science, Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院数字地球科学重点实验室）；University of Chinese Academy of Sciences（中国科学院大学）；Hong Kong Baptist University（香港浸会大学）；Dalian Maritime University（大连海事大学）；Université Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；INRIA（法国国家信息与自动化研究所）；CNRS（法国国家科学研究中心）；Grenoble INP（格勒诺布尔理工学院）；LJK（让·昆兹曼实验室）
Tensor-based methods have been widely studied to attack inverse problems in hyperspectral imaging since a hyperspectral image (HSI) cube can be naturally represented as a third-order tensor, which can perfectly retain the spatial information in the image. In this article, we extend the linear tensor method to the nonlinear tensor method and propose a nonlinear low-rank tensor unmixing algorithm to solve the generalized bilinear model (GBM). Specifically, the linear and nonlinear parts of the GBM can both be expressed as tensors. Furthermore, the low-rank structures of abundance maps and nonlin…
Institut Charles Delaunay (CNRS), Université de Technologie de Troyes（特鲁瓦技术大学，夏尔·德洛内研究所（法国国家科学研究中心））；School of Engineering and Physical Sciences, Heriot-Watt University（赫瑞瓦特大学，工程与物理科学学院）；LITIS lab, Université de Rouen（鲁昂大学，LITIS实验室）
In hyperspectral images, some spectral bands suffer from low signal-to-noise ratio due to noisy acquisition and atmospheric effects, thus requiring robust techniques for the unmixing problem. This paper presents a robust supervised spectral unmixing approach for hyperspectral images. The robustness is achieved by writing the unmixing problem as the maximization of the correntropy criterion subject to the most commonly used constraints. Two unmixing problems are derived: the first problem considers the fully-constrained unmixing, with both the non-negativity and sum-to-one constraints, while th…
University of Missouri（密苏里大学）；University of Florida（佛罗里达大学）
The application of Partial Membership Latent Dirichlet Allocation(PM-LDA) for hyperspectral endmember estimation and spectral unmixing is presented. PM-LDA provides a model for a hyperspectral image analysis that accounts for spectral variability and incorporates spatial information through the use of superpixel-based 'documents.' In our application of PM-LDA, we employ the Normal Compositional Model in which endmembers are represented as Normal distributions to account for spectral variability and proportion vectors are modeled as random variables governed by a Dirichlet distribution. The use…
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
Hyperspectral unmixing is one of the crucial steps for many hyperspectral applications. The problem of hyperspectral unmixing has proven to be a difficult task in unsupervised work settings where the endmembers and abundances are both unknown. What is more, this task becomes more challenging in the case that the spectral bands are degraded with noise. This paper presents a robust model for unsupervised hyperspectral unmixing. Specifically, our model is developed with the correntropy based metric where the non-negative constraints on both endmembers and abundances are imposed to keep physical s…
Tensor-based methods have recently emerged as a more natural and effective formulation to address many problems in hyperspectral imaging. In hyperspectral unmixing (HU), low-rank constraints on the abundance maps have been shown to act as a regularization which adequately accounts for the multidimensional structure of the underlying signal. However, imposing a strict low-rank constraint for the abundance maps does not seem to be adequate, as important information that may be required to represent fine scale abundance behavior may be discarded. This paper introduces a new low-rank tensor regula…
In this study, we propose a novel framework for hyperspectral unmixing by using an improved deep spectral convolution network (DSCN++) combined with endmember uncertainty. DSCN++ is used to compute high-level representations which are further modeled with Multinomial Mixture Model to estimate abundance maps. In the reconstruction step, a new trainable uncertainty term based on a nonlinear neural network model is introduced to provide robustness to endmember uncertainty. For the optimization of the coefficients of the multinomial model and the uncertainty term, Wasserstein Generative Adversaria…
Unsupervised spectral unmixing consists of representing each observed pixel as a combination of several pure materials called endmembers with their corresponding abundance fractions. Beyond the linear assumption, various nonlinear unmixing models have been proposed, with the associated optimization problems solved either by traditional optimization algorithms or deep learning techniques. Current deep learning-based nonlinear unmixing focuses on the models in additive, bilinear-based formulations. By interpreting the reflection process using the discrete Markov chain, the multilinear mixing mod…
Preprocessing Algorithm Leveraging Geometric Modeling for Scale Correction in Hyperspectral Images for Improved Unmixing Performance
Department of Electrical and Electronic Engineering, University of Peradeniya（珀斯德尼亚大学电子与电气工程系）；School of Electrical and Computer Engineering, Purdue University（普渡大学电子与计算机工程学院）
