arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新，休息一下吧，周末愉快~~
arXiv 2503.01628cs.CV

一种同时适用于近端和遥感光谱成像的通用光谱基础模型

A General Purpose Spectral Foundational Model for Both Proximal and Remote Sensing Spectral Imaging

Technical University of Denmark（丹麦技术大学）；University of Copenhagen（哥本哈根大学）

William Michael Laprade, Jesper Cairo Westergaard, Svend Christensen, Mads Nielsen, Anders Bjorholm Dahl

阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对现有基础模型难以处理多通道光谱数据的问题，提出基于掩码自编码器架构的大规模光谱基础模型与数据集，结合光谱通道编码、空间-光谱掩码和ImageNet预训练，适用于近端与遥感光谱成像的下游任务。

摘要：

通过多光谱和高光谱相机获取的光谱成像数据可包含数百个通道，其中每个通道记录特定波长和带宽下的反射率。时间和资源限制了我们收集大型光谱数据集的能力，使得从头构建和训练预测模型变得困难。在RGB领域，我们通常可以通过使用预训练的基础模型作为起点来缓解较小数据集的某些限制。然而，大多数现有基础模型是在3通道RGB图像的大型数据集上预训练的，这严重限制了它们在光谱成像数据上的有效性。少数确实存在的光谱基础模型通常具有以下两种限制之一：(1) 它们仅在遥感数据上构建和训练，限制了其在近端光谱成像中的应用；(2) 它们利用更广泛可用的少于15个通道的多光谱成像数据集，限制了其在百通道高光谱图像中的使用。为了缓解这些问题，我们提出了一个基于掩码自编码器架构的大规模基础模型和数据集，该架构利用光谱通道编码、空间-光谱掩码和ImageNet预训练，为下游光谱成像任务提供一个适应性强且鲁棒的模型。

英文摘要：

Spectral imaging data acquired via multispectral and hyperspectral cameras can have hundreds of channels, where each channel records the reflectance at a specific wavelength and bandwidth. Time and resource constraints limit our ability to collect large spectral datasets, making it difficult to build and train predictive models from scratch. In the RGB domain, we can often alleviate some of the limitations of smaller datasets by using pretrained foundational models as a starting point. However, most existing foundation models are pretrained on large datasets of 3-channel RGB images, severely limiting their effectiveness when used with spectral imaging data. The few spectral foundation models that do exist usually have one of two limitations: (1) they are built and trained only on remote sensing data limiting their application in proximal spectral imaging, (2) they utilize the more widely available multispectral imaging datasets with less than 15 channels restricting their use with hundred-channel hyperspectral images. To alleviate these issues, we propose a large-scale foundational model and dataset built upon the masked autoencoder architecture that takes advantage of spectral channel encoding, spatial-spectral masking and ImageNet pretraining for an adaptable and robust model for downstream spectral imaging tasks.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑