最先进的阿拉伯语言建模：基于稀疏MoE微调和链式思维蒸馏
State-of-the-Art Arabic Language Modeling with Sparse MoE Fine-Tuning and Chain-of-Thought Distillation
AI总结：
本文提出阿拉伯-DeepSeek-R1，通过稀疏MoE架构和链式思维蒸馏，在开放阿拉伯LLM排行榜上实现SOTA，展示了稀疏MoE与文化导向的蒸馏方法在阿拉伯语言任务中的优势。
摘要：
本文介绍了阿拉伯-DeepSeek-R1，一个应用驱动的开源阿拉伯大语言模型，利用稀疏MoE架构解决欠代表语言的数字公平性问题，并在开放阿拉伯LLM排行榜（OALL）上建立新的SOTA。我们的四阶段CoT蒸馏方案整合了阿拉伯特定的语言验证和区域伦理规范，形成一个受控的80/20阿拉伯-英语训练混合。阿拉伯-DeepSeek-R1在七个基准OALL套件中取得最高平均分，实现了SOTA或近SOTA，包括在语法聚焦的MadinahQA上超越GPT-5.1和OALL领导者。我们的结果表明，稀疏MoE架构、文化导向的CoT蒸馏与显式阿拉伯语言检查以及战略双语数据编纂的结合，使开源适应模型在大多数评估综合语言特定任务的基准上系统性地超越专有前沿系统GPT-5.1。这些发现表明，阿拉伯在当前LLM生态系统中的表现劣势主要源于欠专业化而非架构限制，参数高效适应开源推理模型可以在无需工业级预训练成本的情况下实现突破性的SOTA性能。阿拉伯-DeepSeek-R1建立了一个经过验证且可复制的框架，展示了稀疏MoE架构的战略、文化导向适应在低资源语言标准化基准上实现记录性性能的可行性和成本效益。
英文摘要：
This paper introduces Arabic-DeepSeek-R1, an application-driven open-source Arabic LLM that leverages a sparse MoE backbone to address the digital equity gap for under-represented languages, and establishes a new SOTA across the entire Open Arabic LLM Leaderboard (OALL). Our four-phase CoT distillation scheme integrates Arabic-specific linguistic verification and regional ethical norms into a 372M-token, contamination-controlled 80/20 Arabic-English training mixture. Arabic-DeepSeek-R1 achieves the highest average score across the seven-benchmark OALL suite while establishing SOTA or near-SOTA, including dominant results on grammar-focused MadinahQA (surpassing both GPT-5.1 and the OALL leader by substantial margins), safety-oriented AraTrust, multi-ability AlGhafa, and retrieval-augmented ALRAGE. Our results indicate that the combination of sparse MoE architecture, culturally-informed CoT distillation with explicit Arabic linguistic checks, and strategic bilingual data curation enables an open-source adapted model to systematically outperform the proprietary frontier system GPT-5.1 on the majority of benchmarks evaluating comprehensive language-specific tasks: the first such demonstration for Arabic LLMs. These findings indicate that much of Arabic's performance deficit in current LLM ecosystems stems from under-specialization rather than architectural limitations, and that parameter-efficient adaptation of open reasoning models can yield breakthrough SOTA performance without industrial-scale pretraining costs. Arabic-DeepSeek-R1 establishes a validated and replicable framework for sovereign and domain-specific language technologies, demonstrating that strategic, culturally-grounded adaptation of sparse MoE backbones offers a viable and cost-effective pathway to achieving record-breaking performance across standardized benchmarks for low-resource languages.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
