Gipsa-lab（Gipsa实验室）；ONERA（法国航空航天实验室）；Instituto de Telecomunicações（电信研究所）；Instituto Superior Técnico（高等技术学院）；Universidade de Lisboa（里斯本大学）；University of Toulouse（图卢兹大学）；Ghent University（根特大学）；North Atlantic Treaty Organization (NATO) Science and Technology Organization (STO) Centre for Maritime Research and Experimentation (CMRE)（北大西洋公约组织（NATO）科学与技术组织（STO）海事研究与实验中心（CMRE））；University of Tokyo（东京大学）
Laetitia Loncan, Luis B. Almeida, José M. Bioucas-Dias, Xavier Briottet, Jocelyn Chanussot, Nicolas Dobigeon, Sophie Fabre, Wenzhi Liao, Giorgio A. Licciardi, M… 展开作者
Laetitia Loncan, Luis B. Almeida, José M. Bioucas-Dias, Xavier Briottet, Jocelyn Chanussot, Nicolas Dobigeon, Sophie Fabre, Wenzhi Liao, Giorgio A. Licciardi, Miguel Simões, Jean-Yves Tourneret, Miguel A. Veganzones, Gemine Vivone, Qi Wei, Naoto Yokoya
Pansharpening aims at fusing a panchromatic image with a multispectral one, to generate an image with the high spatial resolution of the former and the high spectral resolution of the latter. In the last decade, many algorithms have been presented in the literature for pansharpening using multispectral data. With the increasing availability of hyperspectral systems, these methods are now being adapted to hyperspectral images. In this work, we compare new pansharpening techniques designed for hyperspectral data with some of the state of the art methods for multispectral pansharpening, which have been adapted for hyperspectral data. Eleven methods from different classes (component substitution, multiresolution analysis, hybrid, Bayesian and matrix factorization) are analyzed. These methods are applied to three datasets and their effectiveness and robustness are evaluated with widely used performance indicators. In addition, all the pansharpening techniques considered in this paper have been implemented in a MATLAB toolbox that is made available to the community.
Dual Modality Prompted Diffusion Priors for Zero Shot Hyperspectral Pansharpening
School of Artificial Intelligence, Beijing Normal University（北京师范大学人工智能学院）；Peking University（北京大学）；National Center for Applied Mathematics Shenzhen (NCAMS), Southern University of Science and Technology（南方科技大学深圳应用数学国家中心）；Department of Automation, Tsinghua University（清华大学自动化系）；National Research Council of Italy, Institute of Integrated Methodologies for Earth Observation (CNR-IMIOT)（意大利国家研究委员会综合地球观测方法研究所）
Band-wise Hyperspectral Image Pansharpening using CNN Model Propagation
University Federico II（费德里科二世大学）；National Research Council, Institute of Methodologies for Environmental Analysis, CNR-IMAA（国家研究委员会环境分析方法研究所）；National Biodiversity Future Center（国家生物多样性未来中心）；University Parthenope（帕尔特诺珀大学）
Whiting School of Engineering（怀廷工程学院）；The Johns Hopkins University（约翰斯·霍普金斯大学）
Hyperspectral pansharpening aims to synthesize a low-resolution hyperspectral image (LR-HSI) with a registered panchromatic image (PAN) to generate an enhanced HSI with high spectral and spatial resolution. Recently proposed HS pansharpening methods have obtained remarkable results using deep convolutional networks (ConvNets), which typically consist of three steps: (1) up-sampling the LR-HSI, (2) predicting the residual image via a ConvNet, and (3) obtaining the final fused HSI by adding the outputs from first and second steps. Recent methods have leveraged Deep Image Prior (DIP) to up-sample…
Hyperspectral Pansharpening: Critical Review, Tools and Future Perspectives
University Federico II（那不勒斯费德里科二世大学）；National Research Council（国家研究委员会）；National Biodiversity Future Center（国家生物多样性未来中心）；INRIA（法国国家信息与自动化研究所）；CNRS（法国国家科学研究中心）；Grenoble INP（格勒诺布尔理工学院）；University of Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；Escuela Politecnica de Caceres（卡塞雷斯理工学院）；University Parthenope（帕尔特诺佩大学）
Zero-Shot Hyperspectral Pansharpening Using Hysteresis-Based Tuning for Spectral Quality Control
Department of Electrical Engineering and Information Technology, University Federico II（费米二世大学电气工程与信息科技系）；National Research Council, Institute of Methodologies for Environmental Analysis（国家研究 council 环境分析方法研究所）；Department of Engineering, University Parthenope（帕尔泰诺大学工程系）
THAT: Token-wise High-frequency Augmentation Transformer for Hyperspectral Pansharpening
JPMorgan Chase（摩根大通公司）；University of Missouri-Kansas City（密苏里大学-堪萨斯城分校）；Ming Hsieh Department of Electrical and Computer Engineering（明希赫电气与计算机工程系）；University of Southern California（南加州大学）；Robinson Research Institute（罗宾逊研究学院）；KTH Royal Institute of Technology（皇家理工学院）；NEC Laboratories America（NEC美国实验室）；University of Virginia（弗吉尼亚大学）
Recently, convolutional neural networks (CNN) have obtained promising results in single-image SR for hyperspectral pansharpening. However, enhancing CNNs' representation ability with fewer parameters and a shorter prediction time is a challenging and critical task. In this paper, we propose a novel multi-spectral image fusion method using a combination of the previously proposed 3D CNN model VolumeNet and 2.5D texture transfer method using other modality high resolution (HR) images. Since a multi-spectral (MS) image consists of several bands and each band is a 2D image slice, MS images can be …
Pansharpening aims to fuse a registered high-resolution panchromatic image (PAN) with a low-resolution hyperspectral image (LR-HSI) to generate an enhanced HSI with high spectral and spatial resolution. Existing pansharpening approaches neglect using an attention mechanism to transfer HR texture features from PAN to LR-HSI features, resulting in spatial and spectral distortions. In this paper, we present a novel attention mechanism for pansharpening called HyperTransformer, in which features of LR-HSI and PAN are formulated as queries and keys in a transformer, respectively. HyperTransformer c…
Southwest Jiaotong University（西南交通大学）；Mississippi State University（密西西比州立大学）；University of New South Wales（新南威尔士大学）；University of Extremadura（埃斯特雷马杜拉大学）
Hyperspectral unmixing has been an important technique that estimates a set of endmembers and their corresponding abundances from a hyperspectral image (HSI). Nonnegative matrix factorization (NMF) plays an increasingly significant role in solving this problem. In this article, we present a comprehensive survey of the NMF-based methods proposed for hyperspectral unmixing. Taking the NMF model as a baseline, we show how to improve NMF by utilizing the main properties of HSIs (e.g., spectral, spatial, and structural information). We categorize three important development directions including con…
ONERA（法国航空航天研究院）；University Paris Saclay（巴黎-萨克雷大学）；Univ. Bretagne Sud（南布列塔尼大学）；IRISA（法国信息与随机系统研究所）
In recent years, deep learning techniques revolutionized the way remote sensing data are processed. Classification of hyperspectral data is no exception to the rule, but has intrinsic specificities which make application of deep learning less straightforward than with other optical data. This article presents a state of the art of previous machine learning approaches, reviews the various deep learning approaches currently proposed for hyperspectral classification, and identifies the problems and difficulties which arise to implement deep neural networks for this task. In particular, the issues…
Purdue University Northwest（普渡大学西北校区）；Michigan Technological University（密歇根理工大学）
Deep learning algorithms have seen acute growth of interest in their applications throughout several fields of interest in the last decade, with medical hyperspectral imaging being a particularly promising domain. So far, to the best of our knowledge, there is no review paper that discusses the implementation of deep learning for medical hyperspectral imaging, which is what this review paper aims to accomplish by examining publications that currently utilize deep learning to perform effective analysis of medical hyperspectral imagery. This paper discusses deep learning concepts that are releva…
Deep Learning Techniques for Hyperspectral Image Analysis in Agriculture: A Review
University of Salento（萨伦托大学）；National Research Council of Italy（意大利国家研究委员会）；University Paris-Est Creteil（巴黎东克雷泰伊大学）；Universite Polytechnique Hauts de France（上法兰西理工大学）；Universite de Lille（里尔大学）；CNRS（法国国家科学研究中心）
Hyperspectral unmixing (HU) is a very useful and increasingly popular preprocessing step for a wide range of hyperspectral applications. However, the HU research has been constrained a lot by three factors: (a) the number of hyperspectral images (especially the ones with ground truths) are very limited; (b) the ground truths of most hyperspectral images are not shared on the web, which may cause lots of unnecessary troubles for researchers to evaluate their algorithms; (c) the codes of most state-of-the-art methods are not shared, which may also delay the testing of new methods. Accordingly, t…
Institute of Innovation, Science and Sustainability, Federation University Australia（澳大利亚联邦大学创新、科学与可持续发展研究院）
Wheat is one of the major staple crops across the globe. Therefore, it is mandatory to measure, maintain and improve the wheat quality for human consumption. Traditional wheat quality measurement methods are mostly invasive, destructive and limited to small samples of wheat. In a typical supply chain of wheat, there are many receival points where bulk wheat arrives, gets stored and forwarded as per the requirements. In this receival points, the application of traditional quality measurement methods is difficult and often very expensive. Therefore, there is a need for non-invasive, non-destruct…
Diffusion Models for Hyperspectral Image Analysis: A Comprehensive Review
School of Optical-Electrical and Computer Engineering, University of Shanghai for Science and Technology（上海理工大学光学电子与计算机工程学院）；School of Electronics and Electrical Engineering, Shanghai University of Engineering Science（上海工程技术大学电子与电气工程学院）；Medical Artificial Intelligence Lab, The First Hospital of Hebei Medical University, Hebei Medical University（河北医科大学第一医院医学人工智能实验室）；Hangzhou Institute of Technology, xidian University（杭州职业技术学院）
Hyperspectral images show similar statistical properties to natural grayscale or color photographic images. However, the classification of hyperspectral images is more challenging because of the very high dimensionality of the pixels and the small number of labeled examples typically available for learning. These peculiarities lead to particular signal processing problems, mainly characterized by indetermination and complex manifolds. The framework of statistical learning has gained popularity in the last decade. New methods have been presented to account for the spatial homogeneity of images,…
A comprehensive review of 3D convolutional neural network-based classification techniques of diseased and defective crops using non-UAV-based hyperspectral images
College of Electrical and Information Engineering, Hunan University（湖南大学电气与信息工程学院）
Hyperspectral image (HSI) with high spectral resolution often suffers from low spatial resolution owing to the limitations of imaging sensors. Image fusion is an effective and economical way to enhance the spatial resolution of HSI, which combines HSI with higher spatial resolution multispectral image (MSI) of the same scenario. In the past years, many HSI and MSI fusion algorithms are introduced to obtain high-resolution HSI. However, it lacks a full-scale review for the newly proposed HSI and MSI fusion approaches. To tackle this problem,this work gives a comprehensive review and new guideli…
Amazon Web Service（亚马逊云服务）；University of Houston（休斯顿大学）
Deep neural networks have proven to be very effective for computer vision tasks, such as image classification, object detection, and semantic segmentation -- these are primarily applied to color imagery and video. In recent years, there has been an emergence of deep learning algorithms being applied to hyperspectral and multispectral imagery for remote sensing and biomedicine tasks. These multi-channel images come with their own unique set of challenges that must be addressed for effective image analysis. Challenges include limited ground truth (annotation is expensive and extensive labeling i…
University of Chinese Academy of Sciences（中国科学院大学）；Xidian University（西安电子科技大学）；King Abdullah University of Science and Technology（阿卜杜拉国王科技大学）；Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；Princeton University（普林斯顿大学）；Vrije Universiteit Brussel（布鲁塞尔自由大学）
Hyperspectral imaging enables versatile applications due to its competence in capturing abundant spatial and spectral information, which are crucial for identifying substances. However, the devices for acquiring hyperspectral images are expensive and complicated. Therefore, many alternative spectral imaging methods have been proposed by directly reconstructing the hyperspectral information from lower-cost, more available RGB images. We present a thorough investigation of these state-of-the-art spectral reconstruction methods from the widespread RGB images. A systematic study and comparison of …
Hunan University（湖南大学）；Harbin Institute of Technology（哈尔滨工业大学）；Helmholtz-Zentrum Dresden-Rossendorf（亥姆霍兹德累斯顿罗森多夫研究中心）；Helmholtz Institute Freiberg for Resource Technology（亥姆霍兹弗赖贝格资源技术研究所）；University of Iceland（冰岛大学）
Hyperspectral image (HSI) classification has become a hot topic in the field of remote sensing. In general, the complex characteristics of hyperspectral data make the accurate classification of such data challenging for traditional machine learning methods. In addition, hyperspectral imaging often deals with an inherently nonlinear relation between the captured spectral information and the corresponding materials. In recent years, deep learning has been recognized as a powerful feature-extraction tool to effectively address nonlinear problems and widely used in a number of image processing tas…
Rochester Institute of Technology（罗切斯特理工学院）；The Boeing Company（波音公司）
Hyperspectral sensors enable the study of the chemical properties of scene materials remotely for the purpose of identification, detection, and chemical composition analysis of objects in the environment. Hence, hyperspectral images captured from earth observing satellites and aircraft have been increasingly important in agriculture, environmental monitoring, urban planning, mining, and defense. Machine learning algorithms due to their outstanding predictive power have become a key tool for modern hyperspectral image analysis. Therefore, a solid understanding of machine learning techniques hav…
Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）；University of Chinese Academy of Sciences（中国科学院大学）；International Research Center of Big Data for Sustainable Development Goals（可持续发展大数据国际研究中心）
Owing to the rapid development of sensor technology, hyperspectral (HS) remote sensing (RS) imaging has provided a significant amount of spatial and spectral information for the observation and analysis of the Earth's surface at a distance of data acquisition devices, such as aircraft, spacecraft, and satellite. The recent advancement and even revolution of the HS RS technique offer opportunities to realize the full potential of various applications, while confronting new challenges for efficiently processing and analyzing the enormous HS acquisition data. Due to the maintenance of the 3-D HS …
Jalpaiguri Engineering College（贾尔派古里工程学院）；University of Antwerp（安特卫普大学）；Helmholtz-Zentrum Dresden-Rossendorf（亥姆霍兹德累斯顿罗森多夫研究中心）；Helmholtz Institute Freiberg for Resource Technology（亥姆霍兹弗赖贝格资源技术研究所）
Currently, this paper is under review in IEEE. Transformers have intrigued the vision research community with their state-of-the-art performance in natural language processing. With their superior performance, transformers have found their way in the field of hyperspectral image classification and achieved promising results. In this article, we harness the power of transformers to conquer the task of hyperspectral unmixing and propose a novel deep unmixing model with transformers. We aim to utilize the ability of transformers to better capture the global feature dependencies in order to enhanc…
X-RAFT: Cross-Modal Non-Rigid Registration of Blue and White Light Neurosurgical Hyperspectral Images
Department of Surgical \& Interventional Engineering, School of Biomedical Engineering \& Imaging Sciences, King's College London, London SE1 7EH Department of Neurosurgery, King's College London Hospital NHS Foundation Trust, London SE5 9RS, UK. Department of Imaging Chemistry \& Biology, School of Biomedical Engineering \& Imaging Sciences, King's College London, London SE1 7EH, UK.
The Johns Hopkins University（约翰斯·霍普金斯大学）；U.S. Army Research Laboratory（美国陆军研究实验室）
Pixel-wise classification, where each pixel is assigned to a predefined class, is one of the most important procedures in hyperspectral image (HSI) analysis. By representing a test pixel as a linear combination of a small subset of labeled pixels, a sparse representation classifier (SRC) gives rather plausible results compared with that of traditional classifiers such as the support vector machine (SVM). Recently, by incorporating additional structured sparsity priors, the second generation SRCs have appeared in the literature and are reported to further improve the performance of HSI. These p…
College of Computer Science and Software Engineering, Shenzhen University（深圳大学计算机与软件学院）；SZU Branch, Shenzhen Institute of Artificial Intelligence and Robotics for Society（深圳市人工智能与机器人研究院深圳大学分部）；Department of Computer Science and Engineering, Southern University of Science and Technology（南方科技大学计算机科学与工程系）
With the rapid development of deep learning technology and improvement in computing capability, deep learning has been widely used in the field of hyperspectral image (HSI) classification. In general, deep learning models often contain many trainable parameters and require a massive number of labeled samples to achieve optimal performance. However, in regard to HSI classification, a large number of labeled samples is generally difficult to acquire due to the difficulty and time-consuming nature of manual labeling. Therefore, many research works focus on building a deep learning model for HSI c…
A Comprehensive Survey for Hyperspectral Image Classification: The Evolution from Conventional to Transformers and Mamba Models
Department of Computer Science, National University of Computer and Emerging Sciences（计算机科学系，国立计算机与新兴科学大学）；Dipartimento di Matematica e Informatica-MIFT, University of Messina（数学与信息学系-MIFT，墨西拿大学）；School of Computer Science, University of Hull（计算机科学学院，赫尔大学）；Institute of Software Development and Engineering, Innopolis University（软件开发与工程研究所，因诺波利斯大学）；Aerospace Information Research Institute, Chinese Academy of Sciences（航空信息研究所，中国科学院）；Big Geospatial Data Management, Technical University of Munich（大规模地理空间数据管理，慕尼黑技术大学）；Department of Infrastructure Engineering, University of Melbourne（基础设施工程系，墨尔本大学）；Intelligent Control Laboratory, PLA Rocket Force University of Engineering（智能控制实验室，中国人民解放军火箭军工程大学）；Institute of Methodologies for Environmental Analysis, National Research Council（环境分析方法研究所，国家研究理事会）；School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences（电子、电气与通信工程学院，中国科学院大学）
This paper reviews the second challenge on spectral reconstruction from RGB images, i.e., the recovery of whole-scene hyperspectral (HS) information from a 3-channel RGB image. As in the previous challenge, two tracks were provided: (i) a "Clean" track where HS images are estimated from noise-free RGBs, the RGB images are themselves calculated numerically using the ground-truth HS images and supplied spectral sensitivity functions (ii) a "Real World" track, simulating capture by an uncalibrated and unknown camera, where the HS images are recovered from noisy JPEG-compressed RGB images. A new, …
University of València（瓦伦西亚大学）；University of Lausanne（洛桑大学）；SARMAP SA（SARMAP公司）
Defining an efficient training set is one of the most delicate phases for the success of remote sensing image classification routines. The complexity of the problem, the limited temporal and financial resources, as well as the high intraclass variance can make an algorithm fail if it is trained with a suboptimal dataset. Active learning aims at building efficient training sets by iteratively improving the model performance through sampling. A user-defined heuristic ranks the unlabeled pixels according to a function of the uncertainty of their class membership and then the user is asked to prov…
