Superpixel Graph Contrastive Clustering with Semantic-Invariant Augmentations for Hyperspectral Images
School of Software Engineering, Southeast University（东南大学软件学院）；School of Computer Science and Engineering, Southeast University（东南大学计算机科学与工程学院）；Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education, China（新一代人工智能技术及交叉应用关键实验室（东南大学））；Department of Computing and Information Sciences, Saint Francis University（圣弗朗西斯大学计算机与信息科学系）；Department of Computer Science, City University of Hong Kong（香港城市大学计算机科学系）
Hyperspectral images (HSI) clustering is an important but challenging task. The state-of-the-art (SOTA) methods usually rely on superpixels, however, they do not fully utilize the spatial and spectral information in HSI 3-D structure, and their optimization targets are not clustering-oriented. In this work, we first use 3-D and 2-D hybrid convolutional neural networks to extract the high-order spatial and spectral features of HSI through pre-training, and then design a superpixel graph contrastive clustering (SPGCC) model to learn discriminative superpixel representations. Reasonable augmented views are crucial for contrastive clustering, and conventional contrastive learning may hurt the cluster structure since different samples are pushed away in the embedding space even if they belong to the same class. In SPGCC, we design two semantic-invariant data augmentations for HSI superpixels: pixel sampling augmentation and model weight augmentation. Then sample-level alignment and clustering-center-level contrast are performed for better intra-class similarity and inter-class dissimilarity of superpixel embeddings. We perform clustering and network optimization alternatively. Experimental results on several HSI datasets verify the advantages of the proposed SPGCC compared to SOTA methods. Our code is available at https://github.com/jhqi/spgcc.
National University of Defense Technology（国防科技大学）；University of Trento（特伦托大学）
This paper presents a tensor alignment (TA) based domain adaptation method for hyperspectral image (HSI) classification. To be specific, HSIs in both domains are first segmented into superpixels and tensors of both domains are constructed to include neighboring samples from single superpixel. Then we consider the subspace invariance between two domains as projection matrices and original tensors are projected as core tensors with lower dimensions into the invariant tensor subspace by applying Tucker decomposition. To preserve geometric information in original tensors, we employ a manifold regu…
University of Missouri（密苏里大学）；University of Florida（佛罗里达大学）
A map-guided superpixel segmentation method for hyperspectral imagery is developed and introduced. The proposed approach develops a hyperspectral-appropriate version of the SLIC superpixel segmentation algorithm, leverages map information to guide segmentation, and incorporates the semi-supervised Partial Membership Latent Dirichlet Allocation (sPM-LDA) to obtain a final superpixel segmentation. The proposed method is applied to two real hyperspectral data sets and quantitative cluster validity metrics indicate that the proposed approach outperforms existing hyperspectral superpixel segmentati…
S2RC-GCN: A Spatial-Spectral Reliable Contrastive Graph Convolutional Network for Complex Land Cover Classification Using Hyperspectral Images
School of Computer China University of Geosciences Wuhan, China；School of Geography；Information Engineering China University of Geosciences Wuhan, China；School of Management Wuhan University of Science
Istituto Nazionale di Fisica Nucleare（意大利国家核物理研究所）；Università Milano−Bicocca（米兰比可卡大学）
In this paper, we propose an unsupervised method for hyperspectral remote sensing image segmentation. The method exploits the mean-shift clustering algorithm that takes as input a preliminary hyperspectral superpixels segmentation together with the spectral pixel information. The proposed method does not require the number of segmentation classes as input parameter, and it does not exploit any a-priori knowledge about the type of land-cover or land-use to be segmented (e.g. water, vegetation, building etc.). Experiments on Salinas, SalinasA, Pavia Center and Pavia University datasets are carri…
Joint Superpixel and Self-Representation Learning for Scalable Hyperspectral Image Clustering
Department of Telecommunications and Information Processing（电信与信息处理系）；Ghent University（根特大学）；Royal Institute for Cultural Heritage (KIK-IRPA)（皇家文化遗产研究院）；School of Computer Science（计算机科学学院）；China University of Geosciences（中国地质大学）
Graph-Weighted Contrastive Learning for Semi-Supervised Hyperspectral Image Classification
School of Information Science and Engineering, Lanzhou University（兰州大学信息科学与工程学院）；Key Laboratory of AI and Information Processing, Hechi University（河池学院人工智能与信息处理重点实验室）；School of Artificial Intelligence and Smart Manufacturing, Hechi University（河池学院人工智能与智能制造学院）
In this paper, we propose an unified hyperspectral image classification method which takes three-dimensional hyperspectral data cube as an input and produces a classification map. In the proposed method, a deep neural network which uses spectral and spatial information together with residual connections, and pixel affinity network based segmentation-aware superpixels are used together. In the architecture, segmentation-aware superpixels run on the initial classification map of deep residual network, and apply majority voting on obtained results. Experimental results show that our propoped meth…
An efficient spatial regularization method using superpixel segmentation and graph Laplacian regularization is proposed for sparse hyperspectral unmixing method. Since it is likely to find spectrally similar pixels in a homogeneous region, we use a superpixel segmentation algorithm to extract the homogeneous regions by considering the image boundaries. We first extract the homogeneous regions, which are called superpixels, then a weighted graph in each superpixel is constructed by selecting $K$-nearest pixels in each superpixel. Each node in the graph represents the spectrum of a pixel and edg…
Semi-Supervised Hyperspectral Image Classification with Edge-Aware Superpixel Label Propagation and Adaptive Pseudo-Labeling
Liaoning Technical University（辽宁技术大学）；State Key Laboratory of Regional and Urban Ecology, Institute of Urban Environment, Chinese Academy of Sciences（区域与城市生态国家重点实验室，城市环境研究所，中国科学院）
Harbin Institute of Technology（哈尔滨工业大学）；Wuhan University（武汉大学）；University of Central Florida（中佛罗里达大学）；China University of Geosciences（中国地质大学）
As an unsupervised dimensionality reduction method, principal component analysis (PCA) has been widely considered as an efficient and effective preprocessing step for hyperspectral image (HSI) processing and analysis tasks. It takes each band as a whole and globally extracts the most representative bands. However, different homogeneous regions correspond to different objects, whose spectral features are diverse. It is obviously inappropriate to carry out dimensionality reduction through a unified projection for an entire HSI. In this paper, a simple but very effective superpixelwise PCA approa…
In this paper, we present a graph-based semi-supervised framework for hyperspectral image classification. We first introduce a novel superpixel algorithm based on the spectral covariance matrix representation of pixels to provide a better representation of our data. We then construct a superpixel graph, based on carefully considered feature vectors, before performing classification. We demonstrate, through a set of experimental results using two benchmarking datasets, that our approach outperforms three state-of-the-art classification frameworks, especially when an extremely small amount of la…
Structural-Spectral Graph Convolution with Evidential Edge Learning for Hyperspectral Image Clustering
Southeast University（东南大学）；School of Software Engineering（软件工程学院）；School of Computer Science and Engineering（计算机科学与工程学院）；Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University)（新一代人工智能技术及其交叉应用关键实验室（东南大学））；Department of Computing and Information Sciences（计算与信息科学系）；Department of Computer Science（计算机科学系）
City University of Hong Kong（香港城市大学）；Southeast University（东南大学）；Northwestern Polytechnical University（西北工业大学）；Mississippi State University（密西西比州立大学）
In this paper, we propose a novel classification scheme for the remotely sensed hyperspectral image (HSI), namely SP-DLRR, by comprehensively exploring its unique characteristics, including the local spatial information and low-rankness. SP-DLRR is mainly composed of two modules, i.e., the classification-guided superpixel segmentation and the discriminative low-rank representation, which are iteratively conducted. Specifically, by utilizing the local spatial information and incorporating the predictions from a typical classifier, the first module segments pixels of an input HSI (or its restora…
A central problem in hyperspectral image classification is obtaining high classification accuracy when using a limited amount of labelled data. In this paper we present a novel graph-based framework, which aims to tackle this problem in the presence of large scale data input. Our approach utilises a novel superpixel method, specifically designed for hyperspectral data, to define meaningful local regions in an image, which with high probability share the same classification label. We then extract spectral and spatial features from these regions and use these to produce a contracted weighted gra…
Graphs naturally lend themselves to model the complexities of Hyperspectral Image (HSI) data as well as to serve as semi-supervised classifiers by propagating given labels among nearest neighbours. In this work, we present a novel framework for the classification of HSI data in light of a very limited amount of labelled data, inspired by multi-view graph learning and graph signal processing. Given an a priori superpixel-segmented hyperspectral image, we seek a robust and efficient graph construction and label propagation method to conduct semi-supervised learning (SSL). Since the graph is para…
Beihang University（北京航空航天大学）；Hangzhou Innovation Institute, Beihang University（北京航空航天大学杭州创新研究院）；Beijing Research Institute of Uranium Geology（北京铀矿地质研究所）
Hyperspectral image classification (HIC) is an important but challenging task, and a problem that limits the algorithmic development in this field is that the ground truths of hyperspectral images (HSIs) are extremely hard to obtain. Recently a handful of HIC methods are developed based on the graph convolution networks (GCNs), which effectively relieves the scarcity of labeled data for deep learning based HIC methods. To further lift the classification performance, in this work we propose a graph convolution network (GCN) based framework for HSI classification that uses two clustering operati…
Superpixelwise Low-rank Approximation based Partial Label Learning for Hyperspectral Image Classification
Southern University of Science and Technology（南方科技大学）；Peng Cheng Laboratory（鹏城实验室）；Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）
Unsupervised learning methods for feature extraction are becoming more and more popular. We combine the popular contrastive learning method (prototypical contrastive learning) and the classic representation learning method (autoencoder) to design an unsupervised feature learning network for hyperspectral classification. Experiments have proved that our two proposed autoencoder networks have good feature learning capabilities by themselves, and the contrastive learning network we designed can better combine the features of the two to learn more representative features. As a result, our method s…
DEVCOM Army Research Laboratory (ARL)（美国陆军作战能力发展司令部陆军研究实验室（ARL））
Recently, self-supervised learning has attracted attention due to its remarkable ability to acquire meaningful representations for classification tasks without using semantic labels. This paper introduces a self-supervised learning framework suitable for hyperspectral images that are inherently challenging to annotate. The proposed framework architecture leverages cross-domain CNN, allowing for learning representations from different hyperspectral images with varying spectral characteristics and no pixel-level annotation. In the framework, cross-domain representations are learned via contrasti…
National Institute of Informatics（国立信息学研究所）；Wuhan University（武汉大学）；National Tsing Hua University（国立清华大学）
Understanding foggy image sequence in the driving scenes is critical for autonomous driving, but it remains a challenging task due to the difficulty in collecting and annotating real-world images of adverse weather. Recently, the self-training strategy has been considered a powerful solution for unsupervised domain adaptation, which iteratively adapts the model from the source domain to the target domain by generating target pseudo labels and re-training the model. However, the selection of confident pseudo labels inevitably suffers from the conflict between sparsity and accuracy, both of whic…
Harbin Institute of Technology（哈尔滨工业大学）；Peng Cheng Laboratory（鹏城实验室）；Wuhan University（武汉大学）；National Institute of Informatics（国立情报学研究所）；University of Central Florida（中佛罗里达大学）
Label information plays an important role in supervised hyperspectral image classification problem. However, current classification methods all ignore an important and inevitable problem---labels may be corrupted and collecting clean labels for training samples is difficult, and often impractical. Therefore, how to learn from the database with noisy labels is a problem of great practical importance. In this paper, we study the influence of label noise on hyperspectral image classification, and develop a random label propagation algorithm (RLPA) to cleanse the label noise. The key idea of RLPA …
SuperCL: Superpixel Guided Contrastive Learning for Medical Image Segmentation Pre-training
Institute of Medical Technology, Peking University Health Science Center, Peking University（北京大学医学部医学技术研究所）；Department of Biomedical Engineering, Peking University（北京大学生物医学工程系）；National Biomedical Imaging Center, Peking University（北京大学国家生物医学成像中心）；Institute of Biomedical Engineering, Shenzhen Bay Laboratory（深圳湾实验室生物医学工程研究所）；Institute of Biomedical Engineering, Peking University Shenzhen Graduate School（北京大学深圳研究生院生物医学工程研究所）
Federal University of Santa Catarina（圣卡塔琳娜联邦大学）；Université Côte d’Azur（蔚蓝海岸大学）；Northeastern University（东北大学）；Catholic University of Pelotas（佩洛塔斯天主教大学）
Spectral variability in hyperspectral images can result from factors including environmental, illumination, atmospheric and temporal changes. Its occurrence may lead to the propagation of significant estimation errors in the unmixing process. To address this issue, extended linear mixing models have been proposed which lead to large scale nonsmooth ill-posed inverse problems. Furthermore, the regularization strategies used to obtain meaningful results have introduced interdependencies among abundance solutions that further increase the complexity of the resulting optimization problem. In this …
Quantum Information-Empowered Graph Neural Network for Hyperspectral Change Detection
National Cheng Kung University（成功大学）；Inria（法国国家信息与自动化研究所）；CNRS（法国国家科学研究中心）；Grenoble INP（格勒诺布尔理工学院）；LJK（让·昆廷诺实验室）；Université Grenoble Alpes（格勒诺布尔阿尔卑斯大学）
China University of Geosciences（中国地质大学）；Helmholtz-Zentrum Dresden-Rossendorf（德累斯顿罗森多夫亥姆霍兹中心）；Helmholtz Institute Freiberg for Resource Technology（弗赖贝格资源技术亥姆霍兹研究所）；Institute of Advanced Research in Artificial Intelligence（人工智能高级研究院）；National University of Defense Technology（国防科技大学）
Clustering of hyperspectral images is a fundamental but challenging task. The recent development of hyperspectral image clustering has evolved from shallow models to deep and achieved promising results in many benchmark datasets. However, their poor scalability, robustness, and generalization ability, mainly resulting from their offline clustering scenarios, greatly limit their application to large-scale hyperspectral data. To circumvent these problems, we present a scalable deep online clustering model, named Spectral-Spatial Contrastive Clustering (SSCC), based on self-supervised learning. S…
Hyperspectral imaging is an important sensing technology with broad applications and impact in areas including environmental science, weather, and geo/space exploration. One important task of hyperspectral image (HSI) processing is the extraction of spectral-spatial features. Leveraging on the recent-developed graph signal processing over multilayer networks (M-GSP), this work proposes several approaches to HSI segmentation based on M-GSP feature extraction. To capture joint spectral-spatial information, we first customize a tensor-based multilayer network (MLN) model for HSI, and define a MLN…
A Universal Knowledge Embedded Contrastive Learning Framework for Hyperspectral Image Classification
School of Resource and Environmental Sciences, Wuhan University（武汉大学资源与环境科学学院）；College of Science and Engineering, James Cook University（詹姆斯库克大学科学与工程学院）；School of Computer Science, Wuhan University（武汉大学计算机学院）
German Aerospace Center (DLR)（德国航空航天中心）；Technical University of Munich (TUM)（慕尼黑工业大学）；Beijing Institute of Technology（北京理工大学）；Helmholtz-Zentrum Dresden-Rossendorf（亥姆霍兹德累斯顿罗森多夫研究中心）；Univ. Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；INRIA（法国国家信息与自动化研究所）；CNRS（法国国家科学研究中心）；Grenoble INP（格勒诺布尔国立高等物理、数学与信息学院）
Up to the present, an enormous number of advanced techniques have been developed to enhance and extract the spatially semantic information in hyperspectral image processing and analysis. However, locally semantic change, such as scene composition, relative position between objects, spectral variability caused by illumination, atmospheric effects, and material mixture, has been less frequently investigated in modeling spatial information. As a consequence, identifying the same materials from spatially different scenes or positions can be difficult. In this paper, we propose a solution to addres…
Ocean University of China（中国海洋大学）；Southwest Jiaotong University（西南交通大学）；Mississippi State University（密西西比州立大学）
The joint hyperspectral image (HSI) and LiDAR data classification aims to interpret ground objects at more detailed and precise level. Although deep learning methods have shown remarkable success in the multisource data classification task, self-supervised learning has rarely been explored. It is commonly nontrivial to build a robust self-supervised learning model for multisource data classification, due to the fact that the semantic similarities of neighborhood regions are not exploited in existing contrastive learning framework. Furthermore, the heterogeneous gap induced by the inconsistent …
Hyperspectral Image Classification via Efficient Global Spectral Supertoken Clustering
Beijing Institute of Technology, Beijing 100081, China（北京理工大学）；Beijing Institute of Technology Chongqing Innovation Center, Chongqing 401135, China（北京理工大学重庆创新中心）；Key Laboratory of Photoelectronic Imaging Technology and System, Ministry of Education of China, Beijing 100081, China（光电成像技术与系统重点实验室）
Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Guangzhou University（广州大学）
We introduce a novel video-rate hyperspectral imager with high spatial, and temporal resolutions. Our key hypothesis is that spectral profiles of pixels in a super-pixel of an oversegmented image tend to be very similar. Hence, a scene-adaptive spatial sampling of an hyperspectral scene, guided by its super-pixel segmented image, is capable of obtaining high-quality reconstructions. To achieve this, we acquire an RGB image of the scene, compute its super-pixels, from which we generate a spatial mask of locations where we measure high-resolution spectrum. The hyperspectral image is subsequently…
Beijing Institute of Technology（北京理工大学）；Tsinghua University（清华大学）；Mississippi State University（密西西比州立大学）
Currently, cross-scene hyperspectral image (HSI) classification has drawn increasing attention. It is necessary to train a model only on source domain (SD) and directly transferring the model to target domain (TD), when TD needs to be processed in real time and cannot be reused for training. Based on the idea of domain generalization, a Single-source Domain Expansion Network (SDEnet) is developed to ensure the reliability and effectiveness of domain extension. The method uses generative adversarial learning to train in SD and test in TD. A generator including semantic encoder and morph encoder…
Beijing Institute of Technology（北京理工大学）；The University of Hong Kong（香港大学）
Text information including extensive prior knowledge about land cover classes has been ignored in hyperspectral image classification (HSI) tasks. It is necessary to explore the effectiveness of linguistic mode in assisting HSI classification. In addition, the large-scale pre-training image-text foundation models have demonstrated great performance in a variety of downstream applications, including zero-shot transfer. However, most domain generalization methods have never addressed mining linguistic modal knowledge to improve the generalization performance of model. To compensate for the inadeq…
Benefited from the rich and detailed spectral information in hyperspectral images (HSI), HSI offers great potential for a wide variety of medical applications such as computational pathology. But, the lack of adequate annotated data and the high spatiospectral dimensions of HSIs usually make classification networks prone to overfit. Thus, learning a general representation which can be transferred to the downstream tasks is imperative. To our knowledge, no appropriate self-supervised pre-training method has been designed for histopathology HSIs. In this paper, we introduce an efficient and effe…
Subspace Clustering on Incomplete Data with Self-Supervised Contrastive Learning
Department of Electrical Engineering, Biostatistics（电气工程与生物统计学系）；Wisconsin Institute of Discovery（威斯康星发现研究所）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）
Multi-level Relation Learning for Cross-domain Few-shot Hyperspectral Image Classification
Henan University（河南大学）；Henan Key Laboratory of Big Data Analysis and Processing（河南省大数据分析与处理重点实验室）；Henan Engineering Laboratory of Spatial Information Processing（河南省空间信息处理工程实验室）；Henan Industrial Technology Academy of Spatio-Temporal Big Data（河南省时空大数据产业技术研究院）
