arXiv 2605.172862026-05cs.CV

HyperVision: 一种通道自适应的地基高光谱视觉预训练骨干网络

HyperVision: A Channel-Adaptive Ground-Based Hyperspectral Vision Pre-trained Backbone

Griffith University, Australia（格里菲斯大学，澳大利亚）；Wuhan University, China（武汉大学，中国）；Nanjing University of Science and Technology, China（南京理工大学，中国）；Huaiyin Normal University, China（淮阴师范学院，中国）；Massey University, New Zealand（马斯sey大学，新西兰）

Guanyiman Fu, Jingtao Li, Zihang Cheng, Zhuanfeng Li, Diqi Chen, Yan Xu, Xiangyu Liu, Fengchao Xiong, Jianfeng Lu, Chengrong Chen, Jun Zhou

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对地基高光谱传感器配置差异、标签稀缺与不一致、数据集规模有限等问题，提出首个地基高光谱预训练骨干HyperVision，采用通道自适应动态嵌入、多源伪标签和跨模态知识蒸馏，在三个下游任务上取得最优性能。

摘要：

虽然高光谱成像通过数百个窄波长波段提供丰富的空间-光谱信息，用于精确的材料识别，但地基高光谱预训练骨干网络仍然缺失，受限于传感器间的光谱配置差异、标签的稀缺性和不一致性，以及现有数据集的规模有限和场景多样性不足。为了解决这些挑战并实现通用感知，我们提出了HyperVision，这是首个地基高光谱预训练骨干网络。首先，为了处理不同的光谱配置，HyperVision采用通道自适应动态嵌入机制，将异构输入映射到统一的标记空间。其次，我们开发了一个无监督表示学习框架。具体来说，为了解决标签稀缺和不一致问题，引入了一种多源伪标签方法，融合来自SAM2的空间结构和来自HyperFree的细粒度光谱材料信息。此外，为了丰富场景多样性并补偿有限的数据集规模，利用跨模态知识蒸馏机制，将预训练RGB视觉模型的丰富语义表示迁移到我们的骨干网络。HyperVision在来自26个不同地基数据集的15000张图像集合上进行预训练，展现出卓越的泛化能力。仅需高效的头适配而无需调整骨干参数，它在不同传感器配置下的三个下游任务中取得了比任务特定方法更优的性能，在高光谱语义分割中$\mathrm{Acc}_{\mathrm{M}}$相对提升高达16.3%，目标跟踪AUC相对提升2.1%，显著目标检测MAE降低35.5%。源代码和预训练模型将在https://github.com/lronkitty/HyperVision 公开。

英文摘要：

While hyperspectral imaging provides rich spatial-spectral information across hundreds of narrow wavelength bands for precise material identification, ground-based hyperspectral pre-trained backbones remain absent, constrained by varying spectral configurations across sensors, limited annotations and heterogeneous labeling schemes, and the limited scale and scene diversity of existing datasets. To address these challenges and enable universal perception, we propose HyperVision, the first ground-based hyperspectral pre-trained backbone. First, to handle varying spectral configurations, HyperVision adopts a channel-adaptive dynamic embedding mechanism to map heterogeneous inputs into a unified token space. Second, we develop an unsupervised representation learning framework. Specifically, to address limited annotations and heterogeneous labeling schemes, a multi-source pseudo-labeling method is introduced to fuse spatial structures from SAM2 and fine-grained spectral material information from HyperFree. Furthermore, to enrich scene diversity and compensate for limited dataset scale, a cross-modal knowledge distillation mechanism is utilized to transfer rich semantic representations from a pre-trained RGB vision model to our backbone. Pre-trained on a collection of 15k images from 26 diverse ground-based datasets, HyperVision demonstrates exceptional generalization. Requiring only efficient head-only adaptation without adjusting backbone parameters, it outperforms state-of-the-art task-specific methods across three downstream tasks under varying sensor configurations, yielding up to a 16.3% relative improvement in hyperspectral semantic segmentation $\mathrm{Acc}_{\mathrm{M}}$, a 2.1% relative gain in object tracking AUC, and a 35.5% reduction in salient object detection MAE. The source code and pre-trained models are available at https://github.com/lronkitty/HyperVision.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

AI总结：

摘要：

英文摘要：

相关论文