arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2606.20587cs.NIcs.AIcs.LG
收藏

协议感知分词与架构协同设计用于无线数据包基础模型

Protocol-Aware Tokenization and Architecture Co-Design for Wireless Packet Foundation Models

Cisco Systems(思科系统)

Swadhin Pradhan, Shazal Irshad, Jerome Henry

AI总结:

通过2x2对比实验发现,协议感知分词是无线数据包基础模型性能的主要杠杆,改变分词器可使准确率波动32个百分点,而改变架构仅影响2个百分点。

摘要:

对于构建无线数据包轨迹的基础模型,什么更重要:分词器还是架构,还是两者?为了回答这个问题,我们基于PLUME Anonymous [2026](该工作引入了针对802.11轨迹的协议感知分词)进行扩展;我们扩展了模型深度,并将相同的分词器迁移到一个根本不同的架构家族。一个更深的GPT(PLUME-DEEP,24层)达到了98.2%的top-1准确率,比原始12层设计提升了32个百分点,而Mamba-2状态空间变体(PLUME-MAMBA)在1.7倍吞吐量和2倍上下文长度下实现了96.1%的准确率。关键见解来自跨分词器和架构的受控2x2比较:改变分词器使准确率波动32个百分点;改变架构仅移动2个百分点。协议感知分词是主要的性能杠杆,而骨干网络成为权衡准确率与速度的部署旋钮。

英文摘要:

What matters more for building foundation models for wireless packet traces: the tokenizer or the architecture or both? To answer this question, we build on PLUME Anonymous [2026], which introduced protocol-aware tokenization for 802.11 traces; we scale model depth and transfer the same tokenizer to a fundamentally different architecture family. A deeper GPT (PLUME-DEEP, 24 layers) reaches 98.2% top-1 accuracy, gaining 32 points over the original 12-layer design, while a Mamba-2 state-space variant (PLUME-MAMBA) achieves 96.1% with 1.7x higher throughput and 2x longer context. The key insight emerges from a controlled 2x2 comparison across tokenizers and architectures: changing the tokenizer swings accuracy by 32 points; changing the architecture moves it by only 2. Protocol-aware tokenization is the primary performance lever, and the backbone becomes a deployment knob trading accuracy for speed.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑