自监督表示学习：从光谱基础模型到极光发射光谱
Self-Supervised Representation Learning: From Spectral Foundation Models to Auroral Emission Spectra
浏览论文内容
中文总结 AI 辅助
本文提出用掩码自编码器预训练一维视觉Transformer于极光光谱，无标签下恢复物理特征，微调后超越监督分类器，并探讨现有光谱基础模型的迁移效果。
中文摘要 AI 辅助
极光光谱仪，如Skibotn极光光谱仪（ASIS），记录了数十万条发射光谱，但其中只有几百条能由专家标注。为利用其余数据，我们在223,000条未标注光谱上，用掩码自编码器预训练了一个一维视觉Transformer。在没有标签的情况下，其表示恢复了物理学家用于诊断沉降粒子的发射线强度比（R^2 0.91，而未训练对照组为0.77），并且在一个线性探针下，其分类效果与专家设计的13个特征相当。微调后，该模型在自身基准上超越了之前的监督极光分类器（宏平均精度88.5对77.8），达到0.870 mAP，并且仅用10%的标签就比从头训练的相同架构高出+0.159；归因分析显示它同时使用了N2+谱带。现有的预训练模型能否替代它？两个天文光谱基础模型和一个时间序列模型根据其光谱窗口进行迁移：在红外训练的SpectraFM，性能低于未训练对照组，而在光学训练的SpecFormer，接近域内预训练但未达到。
英文摘要
Auroral spectrographs such as the Auroral Spectrograph In Skibotn (ASIS) record hundreds of thousands of emission spectra, but only a few hundred can be labelled by an expert. To exploit the rest, we pretrain a 1D Vision Transformer with a masked autoencoder on 223,000 unlabelled spectra. Without labels, its representation recovers the emission-line intensity ratios that physicists use to diagnose the precipitating particles (R^2 0.91 vs. 0.77 for an untrained control) and, under one linear probe, classifies as well as 13 features designed by experts. Fine-tuned, the model outperforms the previous supervised auroral classifier on its own benchmark (macro-AP 88.5 vs. 77.8), reaches 0.870 mAP, and exceeds the same architecture trained from scratch by +0.159 with 10% of the labels; attribution shows that it uses both N2+ bands. Could an existing pretrained model replace it? Two astronomical spectral foundation models and a time-series model transfer according to their spectral window: SpectraFM, trained in the infrared, falls below the untrained control, whereas SpecFormer, trained in the optical, approaches in-domain pretraining without reaching it.
发表机构
- IRISA, UMR CNRS 6074, Université Bretagne Sud（IRISA，法国国家科学研究中心联合实验室6074，南布列塔尼大学）
- Royal Belgian Institute for Space Aeronomy（比利时皇家空间航空学研究所）
- UiT - The Arctic University of Norway（挪威北极圈大学）
机构由 AI 辅助整理，请以论文原文为准。
