Apple Intelligence 基础语言模型：2025 技术报告
Apple Intelligence Foundation Language Models: Tech Report 2025
AI总结：
本文介绍 Apple Intelligence 的端侧 3B 模型和基于 PT-MoE 的服务器模型，通过架构创新、多语言多模态训练及 Responsible AI 保障，在基准测试中达到或超过同等规模开源模型，并提供 Swift 框架供开发者集成。
摘要：
我们介绍两个多语言、多模态基础语言模型，它们为 Apple 设备和服务中的 Apple Intelligence 功能提供支持：（i）一个 3B 参数的端侧模型，通过 KV-cache 共享和 2-bit 量化感知训练等架构创新针对 Apple silicon 进行了优化；以及（ii）一个可扩展的服务器模型，基于新颖的 Parallel-Track Mixture-of-Experts（PT-MoE）Transformer 构建，该模型结合了 track 并行、mixture-of-experts 稀疏计算以及交错全局-局部注意力，在 Apple 的 Private Cloud Compute 平台上以具有竞争力的成本提供高质量表现。两个模型均在大规模多语言和多模态数据集上进行训练，这些数据来源于负责任的网络爬取、授权语料库和高质量合成数据，随后在一个新的异步平台上通过监督微调和强化学习进一步优化。所得模型支持多种额外语言，同时能够理解图像并执行工具调用。在公开基准测试和人工评估中，服务器模型和端侧模型均达到或超过同等规模的开源基线模型。一个以 Swift 为中心的新 Foundation Models 框架提供了引导式生成、受限工具调用和 LoRA adapter 微调功能，使开发者只需几行代码即可集成这些能力。Apple Intelligence 模型的最新进展植根于我们的 Responsible AI 方法，包括内容过滤和特定地区评估等防护措施，以及我们通过 Private Cloud Compute 等创新保护用户隐私的承诺。
英文摘要：
We introduce two multilingual, multimodal foundation language models that power Apple Intelligence features across Apple devices and services: i a 3B-parameter on-device model optimized for Apple silicon through architectural innovations such as KV-cache sharing and 2-bit quantization-aware training; and ii a scalable server model built on a novel Parallel-Track Mixture-of-Experts PT-MoE transformer that combines track parallelism, mixture-of-experts sparse computation, and interleaved global-local attention to deliver high quality with competitive cost on Apple's Private Cloud Compute platform. Both models are trained on large-scale multilingual and multimodal datasets sourced via responsible web crawling, licensed corpora, and high-quality synthetic data, then further refined with supervised fine-tuning and reinforcement learning on a new asynchronous platform. The resulting models support several additional languages while understanding images and executing tool calls. In public benchmarks and human evaluations, both the server model and the on-device model match or surpass comparably sized open baselines. A new Swift-centric Foundation Models framework exposes guided generation, constrained tool calling, and LoRA adapter fine-tuning, allowing developers to integrate these capabilities with a few lines of code. The latest advancements in Apple Intelligence models are grounded in our Responsible AI approach with safeguards like content filtering and locale-specific evaluation, as well as our commitment to protecting our users' privacy with innovations like Private Cloud Compute.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
