每个激活皆增强:将通用推理器扩展至1万亿开放语言基础模型
Every Activation Boosted: Scaling General Reasoner to 1 Trillion Open Language Foundation
AI总结:
Ling 2.0基于“每个激活增强推理”原则,在统一MoE范式下从160亿扩展至1万亿参数,通过高稀疏架构、推理导向数据及强化微调,实现7倍计算效率提升,并建立推理精度与效率的新帕累托前沿。
摘要:
我们推出Ling 2.0,这是一个面向推理的语言基础模型系列,其构建原则是每一次激活都能增强推理能力。该系列旨在统一的混合专家(MoE)范式下,将参数规模从数百亿扩展至一万亿,强调高稀疏性、跨尺度一致性以及由经验缩放定律指导的效率。该系列包含三个非思考(指令)模型——Ling-mini-2.0、Ling-flash-2.0和Ling-1T——总参数从160亿到1万亿不等,与稠密对应模型相比,实现了高达7倍的活跃计算效率提升。Ling 2.0整合了模型架构、预训练、后训练和基础设施方面的协同创新:采用高稀疏MoE与MTP实现高效推理,面向推理的数据和中期训练CoT激活,基于强化学习的微调(DFT、Evo-CoT),以及全规模FP8训练与细粒度异构流水线。在万亿规模下,Ling-1T建立了推理准确性与计算效率之间新的帕累托前沿,证明稀疏激活在与推理目标正确对齐时,能够实现可扩展且高效的智能。总体而言,Ling 2.0为推进未来推理和思考模型(包括基于同一基础构建的Ring系列)提供了一个连贯、开放且高效的基础。
英文摘要:
We introduce Ling 2.0, a series reasoning-oriented language foundation built upon the principle that every activation boosts reasoning capability. Designed to scale from tens of billions to one trillion parameters under a unified Mixture-of-Experts (MoE) paradigm, Ling 2.0 emphasizes high sparsity, cross-scale consistency, and efficiency guided by empirical scaling laws. The series includes three non-thinking (instruct) models - Ling-mini-2.0, Ling-flash-2.0, and Ling-1T - ranging from 16B to 1T total parameters and achieving up to 7-fold active-compute efficiency compared with dense counterparts. Ling 2.0 integrates coordinated innovations across model architecture, pre-training, post-training, and infrastructure: a high-sparsity MoE with MTP for efficient reasoning, reasoning-oriented data and mid-training CoT activation, reinforcement-based fine-tuning (DFT, Evo-CoT), and full-scale FP8 training with fine-grained heterogeneous pipelines. At the trillion scale, Ling-1T establishes a new Pareto frontier of reasoning accuracy versus computational efficiency, demonstrating that sparse activation, when properly aligned with reasoning objectives, enables scalable and efficient intelligence. Collectively, Ling 2.0 provides a coherent, open, and efficient foundation for advancing future reasoning and thinking models, including the Ring series built upon the same base.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。