arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2606.29640eess.SPcs.AI
收藏

具有早期退出的快速无线基础模型

Fast Wireless Foundation Models with Early-Exits

Department of Electrical and Software Engineering, University of Calgary(卡尔加里大学电气与软件工程系)

Omar Mashaal, Hatem Abou-Zeid

AI总结:

针对无线基础模型计算成本高的问题,提出在冻结编码器上附加轻量级任务头并选择最优退出阶段,实现可变深度推理,减少93% FLOPs并提升未见分布外任务准确率。

摘要:

虽然无线基础模型(FMs)在实现AI原生6G网络方面展现出强大潜力,但其高计算成本仍是部署的关键障碍。高计算成本源于对每个任务刚性执行FM骨干的完整深度,我们证明这一过程不仅效率低下,还可能降低未见分布外(OOD)任务的性能。本文提出一种新颖的早期退出FM框架,在冻结的无线FM编码器的最合适退出阶段附加轻量级、每任务头,实现针对每个任务首选表示深度的可变深度推理。结果表明,这些中间层特征不仅显著加速推理(最多减少93% FLOPs),而且提供更具迁移性的表示,在未见任务上超过完整编码器准确率。我们进一步证明,每个任务采用简单的固定退出策略比传统早期退出策略(根据感知难度将不同样本路由到不同退出点)更有效。

英文摘要:

While wireless foundation models (FMs) are demonstrating strong potential to enable AI-Native 6G networks, their high computational cost remains a critical barrier to deployment. The large computational cost stems from the rigid, full-depth execution of the FM backbone for every task, a process we show is not only inefficient but can also degrade performance on unseen out-of-distribution (OOD) tasks. In this paper, we propose a novel early-exit FM framework that attaches lightweight, per-task heads, at the most appropriate exit-stage of a frozen wireless FM encoder, enabling variable-depth inference tailored to each task's preferred representation depth. Our results demonstrate that these intermediate-layer features not only speed-up inference significantly (up to 93% fewer FLOPs), but also provide more transferable representations that exceed the full encoder accuracy on unseen tasks. We further demonstrate that a simple fixed-exit strategy per task is more effective than traditional early-exiting policies that route different samples to different exits based on their perceived difficulty levels.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑