EigenSR：基于特征图像桥接的预训练RGB学习器用于单幅高光谱图像超分辨率
EigenSR: Eigenimage-Bridged Pre-Trained RGB Learners for Single Hyperspectral Image Super-Resolution
AI总结：
针对单幅高光谱图像超分辨率数据稀缺问题，本文提出EigenSR框架，通过特征图像桥接预训练RGB模型与HSI，结合迭代光谱正则化保持光谱相关性，在空间和光谱指标上均超越SOTA方法。
摘要：
单幅高光谱图像超分辨率（single-HSI-SR）旨在提升单幅输入低分辨率高光谱图像（HSI）的分辨率。由于数据稀缺的瓶颈，单幅HSI超分辨率的发展远远落后于RGB自然图像超分辨率。近年来，RGB超分辨率的研究表明，在大规模基准数据集上预训练的模型能够大幅提升在未见数据上的性能，这或许可以作为HSI的一种补救方案。但是，我们如何将预训练的RGB模型迁移到HSI，以克服数据稀缺的瓶颈？由于预训练RGB模型与HSI在通道数上存在显著差异，模型无法关注沿光谱维度的相关性，从而限制了其在HSI上的利用能力。受HSI空间-光谱解耦的启发，我们提出了一种新框架：首先利用空间分量（即特征图像，eigenimages）对预训练模型进行微调，然后使用迭代光谱正则化（ISR）在未见HSI上进行推断，以保持光谱相关性。我们方法的优势在于：1）我们有效地将预训练RGB模型的空间纹理处理能力注入HSI，同时保持光谱保真度；2）在光谱去相关域中学习可以提高对光谱无关数据的泛化能力；3）我们在特征图像域中的推断自然地利用了HSI的光谱低秩特性，从而降低了复杂度。这项工作通过特征图像弥合了预训练RGB模型与HSI之间的差距，解决了HSI训练数据有限的问题，因此命名为EigenSR。大量实验表明，EigenSR在空间和光谱指标上均优于最先进（SOTA）方法。
英文摘要：
Single hyperspectral image super-resolution (single-HSI-SR) aims to improve the resolution of a single input low-resolution HSI. Due to the bottleneck of data scarcity, the development of single-HSI-SR lags far behind that of RGB natural images. In recent years, research on RGB SR has shown that models pre-trained on large-scale benchmark datasets can greatly improve performance on unseen data, which may stand as a remedy for HSI. But how can we transfer the pre-trained RGB model to HSI, to overcome the data-scarcity bottleneck? Because of the significant difference in the channels between the pre-trained RGB model and the HSI, the model cannot focus on the correlation along the spectral dimension, thus limiting its ability to utilize on HSI. Inspired by the HSI spatial-spectral decoupling, we propose a new framework that first fine-tunes the pre-trained model with the spatial components (known as eigenimages), and then infers on unseen HSI using an iterative spectral regularization (ISR) to maintain the spectral correlation. The advantages of our method lie in: 1) we effectively inject the spatial texture processing capabilities of the pre-trained RGB model into HSI while keeping spectral fidelity, 2) learning in the spectral-decorrelated domain can improve the generalizability to spectral-agnostic data, and 3) our inference in the eigenimage domain naturally exploits the spectral low-rank property of HSI, thereby reducing the complexity. This work bridges the gap between pre-trained RGB models and HSI via eigenimages, addressing the issue of limited HSI training data, hence the name EigenSR. Extensive experiments show that EigenSR outperforms the state-of-the-art (SOTA) methods in both spatial and spectral metrics.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
