arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.11186cs.CLcs.AI
收藏

赋能波斯语大语言模型的指令遵循能力:一种新型数据集与训练方法

Empowering Persian LLMs for Instruction Following: A Novel Dataset and Training Approach

Isfahan University of Technology(伊斯法罕理工大学);Shahid Bahonar University of Kerman(克尔曼沙希德·巴霍纳尔大学);University of Tehran(德黑兰大学);Hunter College, City University of New York(纽约市立大学亨特学院)

Hojjat Mokhtarabadi, Ziba Zamani, Abbas Maazallahi, Mohammad Hossein Manshaei

AI总结:

针对波斯语低资源场景下大语言模型指令遵循能力不足的问题,研究推出综合性指令数据集FarsInstruct与多任务适配框架Co-CoLA,经实验验证二者结合可有效提升波斯语大模型性能,且数据集将持续更新。

摘要:

经过指令微调的大语言模型已在多个领域展现出遵循人类指令的卓越能力,但在许多低资源语言中,其性能仍存在明显不足。为应对这一挑战,我们首先推出FarsInstruct,这是一个综合性指令数据集,旨在专门提升波斯语(一种在全球范围内使用广泛却代表性不足的语言)大语言模型的指令遵循能力。FarsInstruct涵盖多种任务类型与数据集,每个数据集都包含从简单到复杂的人工编写指令,以及译自Public Pool of Prompts(公共提示池)的指令,确保了丰富的语言与文化代表性。此外,我们提出Co-CoLA框架,用于增强经LoRA(低秩适配)微调的模型的多任务适配能力。通过大量实验分析,本研究证明FarsInstruct数据集结合Co-CoLA框架训练,可有效提升波斯语场景下大语言模型的性能。截至撰写本文时,FarsInstruct包含21个不同数据集下的197个模板,我们计划持续更新该数据集,以进一步提升其适用性。

英文摘要:

Instruction-tuned large language models have demonstrated remarkable capabilities in following human instructions across various domains. However, their proficiency remains notably deficient in many low-resource languages. To address this challenge, we begin by introducing FarsInstruct a comprehensive instruction dataset designed to enhance the instruction following ability of large language models specifically for the Persian language a significant yet underrepresented language globally. FarsInstruct encompasses a wide range of task types and datasets, each containing a mix of straightforward to complex manual written instructions, as well as translations from the Public Pool of Prompts, ensuring a rich linguistic and cultural representation. Furthermore, we introduce Co-CoLA, a framework designed to enhance the multi-task adaptability of LoRA-tuned models. Through extensive experimental analyses, our study showcases the effectiveness of the FarsInstruct dataset coupled with training by the Co-CoLA framework, in improving the performance of large language models within the Persian context. As of the current writing, FarsInstruct comprises 197 templates across 21 distinct datasets, and we intend to update it consistently, thus augmenting its applicability.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑