面向指令微调大语言模型的无种子合成数据生成框架：以泰语为例
Seed-Free Synthetic Data Generation Framework for Instruction-Tuning LLMs: A Case Study in Thai
AI总结：
本文针对低资源语言泰语，提出一种无种子数据的合成指令微调框架，通过融合流畅性、多样性和文化语境三个关键属性，仅用5,000条指令即可达到与大规模数据训练的最先进泰语LLMs相当的竞争力。
摘要：
我们提出了一种以数据高效方式为低资源语言指令微调大语言模型（LLMs）的合成数据方法，特别聚焦于泰语。我们确定了有助于提升指令微调数据集有效性的三个关键属性：流畅性、多样性和文化语境。我们提出了一个无种子数据框架，用于生成融合这些关键属性的合成指令微调数据。该框架利用一个LLM生成多样化主题，从维基百科检索相关上下文，并为问答、摘要和对话等多种任务创建指令。实验结果表明，我们表现最佳的合成数据集融合了全部三个关键属性，仅使用5,000条指令即可取得具有竞争力的性能，可与在数十万条指令上训练的最先进泰语LLMs相媲美。我们的代码和数据集已在 https://github.com/parinzee/seed-free-synthetic-instruct 公开。
英文摘要：
We present a synthetic data approach for instruction-tuning large language models (LLMs) for low-resource languages in a data-efficient manner, specifically focusing on Thai. We identify three key properties that contribute to the effectiveness of instruction-tuning datasets: fluency, diversity, and cultural context. We propose a seed-data-free framework for generating synthetic instruction-tuning data that incorporates these essential properties. Our framework employs an LLM to generate diverse topics, retrieve relevant contexts from Wikipedia, and create instructions for various tasks, such as question answering, summarization, and conversation. The experimental results show that our best-performing synthetic dataset, which incorporates all three key properties, achieves competitive performance using only 5,000 instructions when compared to state-of-the-art Thai LLMs trained on hundreds of thousands of instructions. Our code and dataset are publicly available at https://github.com/parinzee/seed-free-synthetic-instruct.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
