netFound:网络基础模型的原理化设计
netFound: Principled Design for Network Foundation Models
AI总结:
netFound通过四个设计原则提升网络基础模型性能,实现高质量表示与隐私保护,在多个基准测试中表现优异。
摘要:
网络基础模型承诺为多样化的流量分析任务提供可重用的表示,但最近的诊断工作揭示了根本问题:模型利用数据集捷径而非学习真实流量模式,产生坍塌的嵌入空间,并无法捕捉塑造现实行为的外生网络条件。我们将这些诊断洞察转化为四个具体的设计原则:协议感知的标记化、操作上下文嵌入、突发流分层注意力和隐私由设计输入,构建netFound网络基础模型,其架构受此失败分析启发。我们预训练netFound于一个十亿标记规模的语料库上,耗时5000个GPU小时,证明其生成高质量表示,具有较低的各向异性,显著更高的与领域专家特征对齐程度,并在外生上下文鉴别任务中达到F1值0.95,而现有最先进的模型得分低于0.62,同时通过排除有效载荷和IP地址来保护隐私。netFound在冻结编码器评估中表现出显著改进,显示预训练嵌入本身携带有用的结构,并在端到端微调设置中保持所有基准测试的领先者。我们发布了完整的开源代码,HuggingFace上三个模型大小的权重,从原始PCAP到下游推断的容器化流程,以及完整的42亿流量预训练数据集,以促进可重复性和进一步研究。
英文摘要:
Network foundation models promise reusable representations for diverse traffic analysis tasks, but recent diagnostic works have revealed fundamental problems: models exploit dataset shortcuts rather than learning genuine traffic patterns, produce collapsed embedding spaces, and fail to capture the exogenous network conditions that shape real-world behavior. We translate these diagnostic insights into four concrete design principles: protocol-aware tokenization, operational context embedding, burst-flow hierarchical attention, and privacy-by-construction input design, and build netFound, a network foundation model whose architecture is motivated by this failure analysis. We pretrain netFound on a billion-token-scale corpus over 5000 GPU hours, and demonstrate that it produces high-quality representations with lower anisotropy, significantly higher alignment with domain-expert features, and an F1 of 0.95 on exogenous context discrimination where existing state-of-the-art models score below 0.62, while preserving privacy by excluding payload and IP addresses. netFound demonstrates significant improvements in frozen-encoder evaluation, showing that pretrained embeddings themselves carry useful structure, and remains the top performer across all benchmarks in end-to-end fine-tuned settings. We release full open-source code, weights for three model sizes on HuggingFace, a containerized pipeline from raw PCAPs to downstream inference, and the full 4.2 billion flows pretraining dataset to facilitate reproducibility and further research.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。