InstructAlign：通过持续跨语言指令微调实现高低资源语言对齐
InstructAlign: High-and-Low Resource Language Alignment via Continual Crosslingual Instruction Tuning
AI总结：
本文提出InstructAlign框架，通过持续跨语言指令微调使大型语言模型在有限平行数据下学习低资源语言，有效防止灾难性遗忘并保持多任务能力。
摘要：
经过指令微调的大型语言模型（LLMs）在各种任务和语言中展现出了卓越的能力。然而，由于可用数据的稀缺性，它们对代表性不足语言的泛化能力有限。此外，直接将新语言适配到指令微调的LLMs可能会导致灾难性遗忘，进而导致多任务能力的丧失。为了解决这一问题，我们提出了InstructAlign，它使用持续跨语言指令微调来使LLMs将新的未见语言与先前学习的高资源语言对齐。我们的结果证明了InstructAlign在使模型能够利用有限平行数据理解低资源语言的同时防止灾难性遗忘方面的有效性。我们的工作有助于语言适配方法的进步，特别是在将指令微调的LLMs适配到代表性不足的语言方面。我们的代码发布在https://github.com/HLTCHKUST/InstructAlign
英文摘要：
Large language models (LLMs) that are tuned with instructions have demonstrated remarkable capabilities in various tasks and languages. However, their ability to generalize to underrepresented languages is limited due to the scarcity of available data. Additionally, directly adapting new languages to instruction-tuned LLMs can result in catastrophic forgetting, which leads to the loss of multitasking ability. To address this issue, we propose InstructAlign which uses continual crosslingual instruction tuning to enable LLMs to align new unseen languages with previously learned high-resource languages. Our results demonstrate the effectiveness of InstructAlign in enabling the model to understand low-resource languages with limited parallel data while preventing catastrophic forgetting. Our work contributes to the advancement of language adaptation methods, particularly for adapting instruction-tuned LLMs to underrepresented languages. Our code is released on https://github.com/HLTCHKUST/InstructAlign
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
