GemMaroc：以极少量数据解锁LLM的达里贾语能力
GemMaroc: Unlocking Darija Proficiency in LLMs with Minimal Data
AI总结：
GemMaroc通过质量优先的对齐策略和LoRA微调，用极少数据（5K指令）和48 GPU小时训练，使Gemma模型在达里贾语上表现优异，同时保持英文和推理能力，实现绿色AI路径。
摘要：
开源大语言模型（LLM）仍将摩洛哥阿拉伯语（Darija）边缘化，迫使从业者要么加装笨重的阿拉伯语适配器，要么牺牲使LLM有用的推理能力。我们证明，一种严格的质量优先于数量的对齐策略，能够以通常计算量的一小部分，在保护骨干模型跨语言推理能力的同时，展现流利的达里贾语。我们将三个紧凑指令套件LIMA 1K、DEITA 6K和TULU 50K翻译成达里贾语，保留20%的英文原文，并添加数学、编程和科学提示。基于5K混合指令训练的LoRA微调Gemma 3-4B，将DarijaMMLU从32.8提升至42.7；加入推理密集的TULU部分后，将其推至47.5，且英文无回退。将相同配方扩展到Gemma 3-27B，产生GemMaroc-27B，其在DarijaMMLU（61.6）上匹配Atlas-Chat，并在达里贾语常识推理上大幅领先，在HellaSwag上得分60.5，而Atlas-Chat为48.4。关键在于，GemMaroc保留了Gemma-27B强大的数学和通用推理能力，在GSM8K和英文基准上仅显示最小波动。整个模型仅用48 GPU小时训练，凸显了通往包容、可持续语言技术的绿色AI路径。我们发布代码、数据和检查点，以推动教育、公共服务和日常数字交互中的达里贾语中心应用。
英文摘要：
Open-source large language models (LLMs) still marginalise Moroccan Arabic (Darija), forcing practitioners either to bolt on heavyweight Arabic adapters or to sacrifice the very reasoning skills that make LLMs useful. We show that a rigorously quality-over-quantity alignment strategy can surface fluent Darija while safeguarding the backbone s cross-lingual reasoning at a sliver of the usual compute. We translate three compact instruction suites LIMA 1 K, DEITA 6 K and TULU 50 K into Darija, preserve 20 of the English originals, and add mathematics, coding and scientific prompts. A LoRA-tuned Gemma 3-4B trained on 5 K mixed instructions lifts DarijaMMLU from 32.8 to 42.7 ; adding the reasoning-dense TULU portion pushes it to 47.5 with no English regression. Scaling the identical recipe to Gemma 3-27B produces GemMaroc-27B, which matches Atlas-Chat on DarijaMMLU (61.6 ) and leaps ahead on Darija commonsense, scoring 60.5 on HellaSwag versus Atlas-Chat s 48.4 . Crucially, GemMaroc retains Gemma-27B s strong maths and general-reasoning ability, showing only minimal movement on GSM8K and English benchmarks. The entire model is trained in just 48 GPU.h, underscoring a Green AI pathway to inclusive, sustainable language technology. We release code, data and checkpoints to spur Darija-centric applications in education, public services and everyday digital interaction.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
