从基础模型到对话模型：日语指令数据集与大语言模型微调
From Base to Conversational: Japanese Instruction Dataset and Tuning Large Language Models
AI总结：
针对非英语指令微调数据集匮乏的问题，研究构建了日语指令数据集，用LoRA微调多语言模型，验证了日语指令微调的有效性，且小模型也能借此提升下游性能，相关资源已公开。
摘要：
指令微调是大语言模型（LLM）实现交互能力的关键。尽管已有大量英文指令微调数据集，但其他语言的相关数据集明显匮乏，且其在非英语语境下的有效性尚未得到充分验证。我们通过对现有数据集进行扩展与筛选，构建了一个日语指令数据集，并将其应用于日语预训练基础模型。我们使用该指令数据集，对现有的日语和英语模型均开展了低秩适配（Low-Rank Adaptation, LoRA）微调，从定量和定性两个维度评估了这些模型。结果证实了日语指令数据集的有效性，同时表明即便规模相对较小的LLM，也能通过指令微调提升下游任务性能。我们的指令数据集、微调模型及实现代码已在线公开。
英文摘要：
Instruction tuning is essential for large language models (LLMs) to become interactive. While many instruction tuning datasets exist in English, there is a noticeable lack in other languages. Also, their effectiveness has not been well verified in non-English languages. We construct a Japanese instruction dataset by expanding and filtering existing datasets and apply the dataset to a Japanese pre-trained base model. We performed Low-Rank Adaptation (LoRA) tuning on both Japanese and English existing models using our instruction dataset. We evaluated these models from both quantitative and qualitative perspectives. As a result, the effectiveness of Japanese instruction datasets is confirmed. The results also indicate that even with relatively small LLMs, performances in downstream tasks would be improved through instruction tuning. Our instruction dataset, tuned models, and implementation are publicly available online.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
