发表机构
University of Calgary（卡尔加里大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出通过目标多样性与多分支架构扩展无线基础模型，在参数更少的情况下提升多个下游任务性能。
AI 中文摘要
无线基础模型从无标注的无线电信号中学习表示，以便在下游任务中重用。缩放模型容量是学习更丰富表示并提升下游性能的常见策略。然而，在预训练数据有限的情况下，其在无线自监督学习中的收益并不稳定。我们研究目标多样性作为另一种缩放轴：不同的自监督目标强调不同的信号特性，组合它们的表示可以保留更多信息以支持多样化的任务。我们开发了一个融合框架，将独立编码器通过重构、预测和对比学习训练得到的冻结表示进行组合。这为多样性的益处提供了参考，但需要维护多个编码器。为了在标准单目标编码器的参数预算内保留这些益处，我们引入了一个联合训练的多分支架构，包含共享主干和特定于目标的分支。我们在包含频谱图和信道状态信息的异构语料库上进行预训练，并评估了涵盖通信、感知和定位的六个下游任务。在相同的输出嵌入维度下，融合在所有六个任务上优于所评估的单目标编码器，同时仅使用其约三分之一的参数。多分支架构在单编码器参数预算内保留了融合的大部分益处。表示分析表明各目标之间存在互补贡献，其中大部分额外益处保留在与重构表示子空间正交的分量中。这些发现支持目标多样性作为扩展无线基础模型的有效策略。
英文摘要
Wireless foundation models learn representations from unlabeled radio signals for reuse across downstream tasks. Scaling model capacity is a common strategy for learning richer representations and improving downstream performance. However, its gains are less consistent in wireless self-supervised learning when pretraining data are limited. We investigate objective diversity as an alternative scaling axis: different self-supervised objectives emphasize different signal properties, and combining their representations can preserve more information to enable diverse tasks. We develop a fusion framework that combines frozen representations from independent encoders trained through reconstructive, predictive, and contrastive learning. This provides a reference for the benefits of diversity, but requires the maintenance of multiple encoders. To retain these benefits within the parameter budget of a standard single-objective encoder, we introduce a jointly trained multi-branch architecture with a shared trunk and objective-specific branches. We pretrain on a heterogeneous corpus of spectrogram and channel state information data and evaluate six downstream tasks that span communication, sensing, and positioning. At an equal output embedding dimension, fusion outperforms the evaluated single-objective encoders on all six tasks while using roughly one-third of their parameters. The multi-branch architecture retains much of the fusion benefit within a single-encoder parameter budget. Representation analyses indicate complementary contributions across objectives, with much of the added benefit retained in components orthogonal to the reconstruction representation subspace. These findings support objective diversity as an effective strategy for scaling wireless foundation models.
