Hyperspectral imaging (HSI) enables detailed land cover classification, yet low spatial resolution and sparse annotations pose significant challenges. We present a label-efficient framework that leverages spatial features from a frozen diffusion model pretrained on natural images. Our approach extracts low-level representations from high-resolution decoder layers at early denoising timesteps, which transfer effectively to the low-texture structure of HSI. To integrate spectral and spatial information, we introduce a lightweight FiLM-based fusion module that adaptively modulates frozen spatial features using spectral cues, enabling robust multimodal learning under sparse supervision. Experiments on two recent hyperspectral datasets demonstrate that our method outperforms state-of-the-art approaches using only the provided sparse training labels. Ablation studies further highlight the benefits of diffusion-derived features and spectral-aware fusion. Overall, our results indicate that pretrained diffusion models can support domain-agnostic, label-efficient representation learning for remote sensing and broader scientific imaging tasks.
Army Research Laboratory（美国陆军研究实验室）；Booz Allen Hamilton Inc.（博思艾伦汉密尔顿公司）
A pretrain-finetune strategy is widely used to reduce the overfitting that can occur when data is insufficient for CNN training. First few layers of a CNN pretrained on a large-scale RGB dataset are capable of acquiring general image characteristics which are remarkably effective in tasks targeted for different RGB datasets. However, when it comes down to hyperspectral domain where each domain has its unique spectral properties, the pretrain-finetune strategy no longer can be deployed in a conventional way while presenting three major issues: 1) inconsistent spectral characteristics among the …
Exploring Multi-Timestep Multi-Stage Diffusion Features for Hyperspectral Image Classification
School of Information Science and Technology, Fudan University（复旦大学信息科学与技术学院）；Academy for Engineering and Technology, Fudan University（复旦大学工程与技术学院）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）
Dual Modality Prompted Diffusion Priors for Zero Shot Hyperspectral Pansharpening
School of Artificial Intelligence, Beijing Normal University（北京师范大学人工智能学院）；Peking University（北京大学）；National Center for Applied Mathematics Shenzhen (NCAMS), Southern University of Science and Technology（南方科技大学深圳应用数学国家中心）；Department of Automation, Tsinghua University（清华大学自动化系）；National Research Council of Italy, Institute of Integrated Methodologies for Earth Observation (CNR-IMIOT)（意大利国家研究委员会综合地球观测方法研究所）
DiffSpectralNet : Unveiling the Potential of Diffusion Models for Hyperspectral Image Classification
University of Buckingham（白金汉大学）；University of Greenwich（格林威治大学）；Middlesex University（密德萨斯大学）；International University–Vietnam National University of HCMC（胡志明市越南国家大学国际大学）
Huazhong University of Science & Technology（华中科技大学）；Wuhan Jingce Electronic Group（武汉精测电子集团）
We present a simple, efficient, and scalable unfolding network, SAUNet, to simplify the network design with an adaptive alternate optimization framework for hyperspectral image (HSI) reconstruction. SAUNet customizes a Residual Adaptive ADMM Framework (R2ADMM) to connect each stage of the network via a group of learnable parameters to promote the usage of mask prior, which greatly stabilizes training and solves the accuracy degradation issue. Additionally, we introduce a simple convolutional modulation block (CMB), which leads to efficient training, easy scale-up, and less computation. Couplin…
GANs have matured in recent years and are able to generate high-resolution, realistic images. However, the computational resources and the data required for the training of high-quality GANs are enormous, and the study of transfer learning of these models is therefore an urgent topic. Many of the available high-quality pretrained GANs are unconditional (like StyleGAN). For many applications, however, conditional GANs are preferable, because they provide more control over the generation process, despite often suffering more training difficulties. Therefore, in this paper, we focus on transferri…
Satellite imagery allows a plethora of applications ranging from weather forecasting to land surveying. The rapid development of computer vision systems could open new horizons to the utilization of satellite data due to the abundance of large volumes of data. However, current state-of-the-art computer vision systems mainly cater to applications that mainly involve natural images. While useful, those images exhibit a different distribution from satellite images in addition to having more spectral channels. This allows the use of pretrained deep learning models only in a subset of spectral chan…
HIR-ALIGN: Enhancing Hyperspectral Image Restoration via Diffusion-Based Data Generation
School of Mathematics and Statistics, Xi’an Jiaotong University（西安交通大学数学与统计学学院）；School of Computer Science and Technology, Xi’an Jiaotong University（西安交通大学计算机科学与技术学院）；School of Mathematics and Statistics and the Ministry of Education Key Laboratory for Intelligent Networks and Network Security, Xi’an Jiaotong University（西安交通大学数学与统计学学院和教育部智能网络与网络安全重点实验室）；Pazhou Laboratory (Huangpu), Guangzhou（广州黄埔 Pazhou 实验室）
Department of Engineering Cybernetics, Norwegian University of Science and Technology（工程 cybernetics 部，挪威科学与技术大学）；Department of Biology, Norwegian University of Science and Technology（生物学部，挪威科学与技术大学）；Department of Automatic Control and Systems Engineering, University Politehnica Bucharest（自动控制与系统工程部，布加勒斯特理工大学）
Physics-Inspired Degradation Models for Hyperspectral Image Fusion
Beijing Institute of Technology（北京理工大学）；Hunan University（湖南大学）；University of Science and Technology of China（中国科学技术大学）；Beijing Normal University（北京师范大学）
PromptHSI: Universal Hyperspectral Image Restoration with Vision-Language Modulated Frequency Adaptation
Institute of Data Science, National Cheng Kung University（成功大学数据科学研究所）；Graduate Institute of Communication Engineering, National Taiwan University（台湾大学通讯工程学系）；NVIDIA（英伟达公司）
UniDiff: Parameter-Efficient Adaptation of Diffusion Models for Land Cover Classification with Multi-Modal Remotely Sensed Imagery and Sparse Annotations
LUM-ViT: Learnable Under-sampling Mask Vision Transformer for Bandwidth Limited Optical Signal Acquisition
College of Information Science & Electronic Engineering, Zhejiang University（浙江大学信息科学与电子工程学院）；College of Control Science and Engineering, Zhejiang University（浙江大学控制科学与工程学院）
Rice University（莱斯大学）；Carnegie Mellon University（卡内基梅隆大学）；Sony Group Corporation（索尼集团公司）
Spatially varying spectral modulation can be implemented using a liquid crystal spatial light modulator (SLM) since it provides an array of liquid crystal cells, each of which can be purposed to act as a programmable spectral filter array. However, such an optical setup suffers from strong optical aberrations due to the unintended phase modulation, precluding spectral modulation at high spatial resolutions. In this work, we propose a novel computational approach for the practical implementation of phase SLMs for implementing spatially varying spectral filters. We provide a careful and systemat…
FUN: A Focal U-Net Combining Reconstruction and Object Detection for Snapshot Spectral Imaging
School of Artificial Intelligence, Xidian University（西安电子科技大学人工智能学院）；Changzhi Medical College（长治医学院）；the Engineering Research Centre for Intelligent Data Assisted Diagnosis and Treatment in Shanxi Province（山西省智能数据辅助诊断与治疗工程研究中心）；Uniwave Artificial Intelligence Technology Co., Ltd.（Uniwave人工智能科技有限公司）
Recent work has shown generative adversarial networks (GANs) can generate highly realistic images, that are often indistinguishable (by humans) from real images. Most images so generated are not contained in the training dataset, suggesting potential for augmenting training sets with GAN-generated data. While this scenario is of particular relevance when there are limited data available, there is still the issue of training the GAN itself based on that limited data. To facilitate this, we leverage existing GAN models pretrained on large-scale datasets (like ImageNet) to introduce additional kn…
Awakening Diffusion Transformers: Eliciting Stronger Generation and Understanding via Massive Activation Modulation
Shanghai Jiao Tong University（上海交通大学）；Monash University（莫纳什大学）；The University of Hong Kong（香港大学）；Zhongguancun Academy（中关村科学城创新中心）；Chulalongkorn University（朱拉隆功大学）；City University of Hong Kong（香港城市大学）
Instruct-IPT: All-in-One Image Processing Transformer via Weight Modulation
State Key Lab of General AI, School of Intelligence Science and Technology, Peking University（北京大学智能科学与技术学院通用人工智能国家重点实验室）；Huawei Noah’s Ark Lab（华为诺亚方舟实验室）
Applied Research Center (ARC), Tencent PCG（腾讯平台与内容事业群应用研究中心）
Colorization has attracted increasing interest in recent years. Classic reference-based methods usually rely on external color images for plausible results. A large image database or online search engine is inevitably required for retrieving such exemplars. Recent deep-learning-based methods could automatically colorize images at a low cost. However, unsatisfactory artifacts and incoherent colors are always accompanied. In this work, we propose GCP-Colorization that leverages the rich and diverse color priors encapsulated in a pretrained Generative Adversarial Networks (GAN) for automatic colo…
Underwater image enhancement has become an attractive topic as a significant technology in marine engineering and aquatic robotics. However, the limited number of datasets and imperfect hand-crafted ground truth weaken its robustness to unseen scenarios, and hamper the application to high-level vision tasks. To address the above limitations, we develop an efficient and compact enhancement network in collaboration with a high-level semantic-aware pretrained model, aiming to exploit its hierarchical feature representation as an auxiliary for the low-level underwater image enhancement. Specifical…
Knowledge Transfer and Domain Adaptation for Fine-Grained Remote Sensing Image Segmentation
School of Computer Technology and Applications, Qinghai University（青海大学计算机技术与应用学院）；School of Computer Science and Technology, Beijing Jiaotong University（北京交通大学计算机科学与技术学院）；Department of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术系）
Recovering Partially Corrupted Objects via Sketch-Guided Bidirectional Feature Interaction
School of Electrical and Data Engineering（电气与数据工程学院）；Australian Artificial Intelligence Institute（澳大利亚人工智能研究所）；University of Technology Sydney（悉尼科技大学）；School of Computer Science and Information Engineering（计算机科学与信息工程学院）；Hefei University of Technology（合肥工业大学）
