发表机构
KI Bundesverband; DFKI; Fraunhofer IAIS; Fraunhofer IIS; Technische Universität Darmstadt; Universität Würzburg; Berliner Hochschule für Technik; L3S Research Center; ellamind; Merantix Momentum（德国人工智能协会; 德国人工智能研究中心; 弗劳恩霍夫自动化与信息研究所; 弗劳恩霍夫集成电路研究所; 达姆施塔特工业大学; 维尔茨堡大学; 柏林技术大学; L3S研究中心; 艾拉明德公司; 梅兰蒂克斯动力公司）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究提出自主开源的Soofi S 30B - A3B基础模型，采用MoE混合设计，预训练于约27万亿令牌，在德英语基准测试表现出色，优于欧洲自主基线及其他开源模型，将在宽松条款下发布，为相关领域提供强大且开源的模型支持。
AI 中文摘要
我们展示了Soofi S 30B - A3B，这是一个用于德语和英语的自主开源专家混合（MoE）混合曼巴变换器基础模型。其混合设计每个令牌仅激活30B参数中的3B，且随着上下文增长推理缓存近乎恒定，在长上下文、高并发部署中比密集模型具有决定性的吞吐量优势。在约27万亿个令牌上预训练，德语权重特意上调，Soofi S在英语和德语综合基准测试中与14B至27B密集模型相当，在17个开源基础模型中两种语言的代码聚合表现最佳，优于所有欧洲自主基线。在完全开源模型中，Soofi S获得最高的英语和德语评估分数。Soofi S在德国工业AI云（由德国电信在慕尼黑运营的自主HPC规模AI基础设施）上全流程构建，并将在高度宽松、开放访问条款下发布。
英文摘要
We present Soofi S 30B-A3B, a sovereign, open-source Mixture-of-Experts (MoE) hybrid Mamba Transformer foundation model for German and English. Its hybrid design activates only 3B of 30B parameters per token and keeps the inference cache near-constant as context grows, giving it a decisive throughput advantage over dense models for long-context, high-concurrency deployment. Pretrained on roughly 27 trillion tokens with deliberately up-weighted German, Soofi S matches dense 14 to 27B models on aggregate English and German benchmarks while achieving the best code aggregates in both languages among 17 open base models, and outperforms every European sovereign baseline in our comparison, including ones far larger in active parameters. Among fully open models, Soofi S obtains the highest English and German evaluation scores, ahead of Olmo 3 32B and Apertus 70B. Soofi S was built end-to-end on the German Industrial AI Cloud, a sovereign HPC scale AI infrastructure operated by Deutsche Telekom in Munich. Soofi S will be released under highly permissive, open-access terms: weights, selected intermediate checkpoints, full per-source data accounting, hyperparameters, and training and evaluation code. Where source licenses permit, data-construction artifacts are released under permissive licenses; commercially licensed sources are documented with aggregate statistics and exact mixture accounting.
