AI 中文总结
Opt.Gear系列含多参数规模模型，采用混合架构优化KV缓存，数据效率高，支持多硬件部署，还推出可在MCU运行的Opt.Gear-1M，为边缘应用提供实用基础。
AI 中文摘要
我们推出Opt.Gear，这是一款专为高效设备端部署、实时推理和强大任务能力设计的基础模型。它包含参数量为100万、2.7亿和10亿的密集模型，上下文长度达64K。我们设计了一种新的混合架构，将卷积键值门控混合器与局部-全局注意力相结合，以减少随长上下文呈指数级增长的KV缓存内存。该架构在NPU上与类似规模的模型相比，预填充和解码速度最高提升4.9倍。我们从2万亿token的候选语料库中，在未使用知识蒸馏的情况下，对精选的5000亿token子集进行了训练，这是现有基础模型中数据效率最高的。所有模型均以开放权重和ONNX、高通NPU、苹果ANE的部署二进制文件形式发布，使Opt.Gear成为需要快速、内存高效推理和强大任务能力的边缘应用的实用基础。此外，为扩设备端生成式语言模型的生态系统，我们推出Opt.Gear-1M，这是一款可部署在微控制器单元（MCU）上的微型语言模型（TLM）。Opt.Gear-1M是首款在STM32H747I-DISCO的ARM Cortex-M7 CPU上以W4A32量化实现20 TPS的生成式语言模型。
英文摘要
We introduce OptGear, a foundation model designed for efficient on-device deployment, real-tim inference, and strong task capability. It includes a dense model (1M, 270M, and 1B) with a context length of 64K. We designed a new hybrid architecture that combines a convolutional key-value gated mixer with local-global attention to reduce the KV-cache memory that tends to increase exponentially with long context. This architecture delivers up to X4.9 faster prefill and decoding speeds on the NPUs compared to models of a similar scale models. From a 2T tokens candidate corpus, OptGear is trained on a curated 0.5T tokens subset without knowledge distillation. This is the most data-efficient of the existing foundation models. All models are released with open weights and deployment binaries for ONNX, Qualcomm NPU, and Apple ANE making OptGear a practical base for edge applications that need fast, memory-efficient inference and strong task capabilities. Furthermore, to expand the ecosystem of on-device generative language models, we are introducing the OptGear-1M that can be deployed on Micro-Controller Units (MCUs), a Tiny Language Model (TLM). OptGear-1M is the first generative language model to achieve 20 TPS with W4A32 quantization on the ARM Cortex-M7 CPU of the STM32H747I-DISCO.
Comments[OptAI] OptGear Model Technical Report
