arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2511.04015eess.SP
收藏

Tiny-WiFo: 一种轻量级无线基础模型用于信道预测的多组件自适应知识蒸馏

Tiny-WiFo: A Lightweight Wireless Foundation Model for Channel Prediction via Multi-Component Adaptive Knowledge Distillation

Peking University(北京大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))

Haotian Zhang, Shijian Gao, Xiang Cheng

AI总结:

Tiny-WiFo通过多组件自适应知识蒸馏方法,在保持高性能的同时实现轻量级无线基础模型的高效部署。

摘要:

无线基础模型(FMs)的庞大规模阻碍了其在边缘设备上的实时部署。本文超越了标准的知识蒸馏,引入了一种新的多组件自适应知识蒸馏(MCAKD)框架。关键创新包括基于交叉注意力的知识选择(CA-KS)模块,该模块从教师模型中选择关键特征,以及自主学习-被动学习(AL-PL)策略,该策略在知识转移与独立学习之间取得平衡,以在可控的计算成本下实现高训练效率。当应用于WiFo FM时,蒸馏后的Tiny-WiFo模型仅包含5.5M个参数,推理时间仅为1.6毫秒,同时保留了超过98%的WiFo性能及其关键的零样本泛化能力,使实时FM部署成为可能。

英文摘要:

The massive scale of Wireless Foundation Models (FMs) hinders their real-time deployment on edge devices. This letter moves beyond standard knowledge distillation by introducing a novel Multi-Component Adaptive Knowledge Distillation (MCAKD) framework. Key innovations include a Cross-Attention-Based Knowledge Selection (CA-KS) module that selectively identifies critical features from the teacher model, and an Autonomous Learning-Passive Learning (AL-PL) strategy that balances knowledge transfer with independent learning to achieve high training efficiency at a manageable computational cost. When applied to the WiFo FM, the distilled Tiny-WiFo model, with only 5.5M parameters, achieves a 1.6 ms inference time while retaining over 98% of WiFo's performance and its crucial zero-shot generalization capability, making real-time FM deployment viable.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑