UrduLLaMA 1.0：低资源环境下的数据集策展、预处理与评估
UrduLLaMA 1.0: Dataset Curation, Preprocessing, and Evaluation in Low-Resource Settings
AI总结：
针对乌尔都语低资源问题，基于Llama-3.1-8B-Instruct持续预训练1.28亿token，并用LoRA微调指令与翻译数据，显著提升性能，建立新基准。
摘要：
多语言大语言模型（LLMs）在乌尔都语等低资源语言上往往表现欠佳。本文介绍了UrduLLaMA 1.0，该模型基于开源Llama-3.1-8B-Instruct架构，并在1.28亿个乌尔都语token上持续预训练，捕捉了该语言的丰富多样性。为了增强指令遵循和翻译能力，我们利用低秩适配（LoRA）技术在41,000条乌尔都语指令和约50,000对英乌翻译对上对模型进行微调。在三个机器翻译数据集上的评估表明，与最先进的（SOTA）模型相比，性能显著提升，为乌尔都语LLM建立了新的基准。这些发现强调了在有限数据和计算资源下，有针对性的适配策略在解决低资源语言特有挑战方面的潜力。
英文摘要：
Multilingual Large Language Models (LLMs) often provide suboptimal performance on low-resource languages like Urdu. This paper introduces UrduLLaMA 1.0, a model derived from the open-source Llama-3.1-8B-Instruct architecture and continually pre-trained on 128 million Urdu tokens, capturing the rich diversity of the language. To enhance instruction-following and translation capabilities, we leverage Low-Rank Adaptation (LoRA) to fine tune the model on 41,000 Urdu instructions and approximately 50,000 English-Urdu translation pairs. Evaluation across three machine translation datasets demonstrates significant performance improvements compared to state-of-the-art (SOTA) models, establishing a new benchmark for Urdu LLMs. These findings underscore the potential of targeted adaptation strategies with limited data and computational resources to address the unique challenges of low-resource languages.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
