WALoMA：一种通过自适应低秩掩码自动编码器实现的多任务无线基础模型

中文摘要 AI 辅助

本文提出了一种通过自适应低秩掩码自动编码器（WALoMA）的多任务无线基础模型，这是一种用于第六代（6G）无线物理层架构的统一多任务基础模型，旨在解决专用的、特定任务深度学习模型的局限性以及稀缺标记无线数据集的实际挑战。该框架利用受基础模型启发的概念，采用掩码自动编码器（MAE）范式从未标记的信道数据中学习，以显著减少对大量注释的依赖。模型将无线信道状态信息（CSI）视为通用模态，并通过自监督信道重建学习可转移表示。关键架构创新包括使用二维位置编码（PE）来明确保留天线和子载波之间的空间频率关系，以及用于参数高效微调的低秩适应（LoRA）。该框架在五个下游任务中展现出有效性，在视距/非视距分类中得分为96.47%，波束预测中为80.哈45%，信道插值中为85.78%，信道估计中为99.12%，信道绘图中为77.18%。数值结果表明，所提出的模型实现了87.80%的综合得分，显著优于大型无线模型（LWM）基线的59.90%，同时平均仅训练总参数的14.68%，甚至在标记数据极其有限的条件下也保持强大性能。

英文摘要

This paper proposes a multitask wireless foundation model via adaptive low-rank masked autoencoders (WALoMA), a unified multi-task foundation model for sixth-generation (6G) wireless physical layer architectures, to address the limitations of specialized, task-specific deep learning models and the practical challenge of scarce labeled wireless datasets. By leveraging concepts inspired by foundation models, the proposed framework adopts a masked autoencoder (MAE) paradigm to learn from unlabeled channel data, to significantly reduce reliance on extensive annotations. The model treats wireless channel state information (CSI) as a universal modality and learns transferable representations through self-supervised channel reconstruction. Key architectural novelties include the use of 2D positional encoding (PE) to explicitly preserve the spatial-frequency relationships between antennas and subcarriers, and low-rank adaptation (LoRA) for parameter-efficient fine-tuning. The framework's efficacy is demonstrated across five downstream tasks, achieving individual scores of 96.47\% for LoS/NLoS classification, 80.45\% for beam prediction, 85.78\% for channel interpolation, 99.12\% for channel estimation, and 77.18\% for channel charting. Consequently, numerical results show that the proposed model achieves a composite score of 87.80\%, significantly outperforming the 59.90\% achieved by the large wireless model (LWM) baseline while training an average of only 14.68\% of total parameters, and maintaining strong performance even under extremely limited labeled data conditions.

发表机构

• King Abdullah University of Science and Technology (KAUST)（阿卜杜拉国王科技大学）
• University of Southampton（南安普顿大学）

机构由 AI 辅助整理，请以论文原文为准。