Cookbook：一种通过程序化数据生成模板提升大语言模型生成能力的框架
Cookbook: A framework for improving LLM generative abilities via programmatic data generating templates
AI总结：
本研究提出Cookbook程序化数据生成框架，通过Python模板生成非人工/LLM的指令训练数据，可提升单任务准确率最高52.7点，多任务上Mistral-7B微调后在GPT4ALL套件中7B级模型平均最优。
摘要：
在指令数据集上微调大语言模型（LLM）是提升其生成能力的常用方法。然而，人工整理指令数据集成本高昂且耗时，而LLM生成的数据虽人力投入较少，却可能违反用户隐私协议或LLM服务商的服务条款。因此，我们寻求一种构建指令数据集的方法，其样本既非人类也非LLM生成，却仍能提升LLM的生成能力。\n本研究提出Cookbook框架，该框架通过程序化方式生成由随机令牌上的简单模式构成的训练数据，形成一种可扩展、高性价比且规避法律与隐私问题的方案。首先，Cookbook使用模板——一种生成数据的Python函数——来生成训练数据，促使模型学习与目标任务对应的、基于显式模式的规则。我们发现，在Cookbook生成的数据上微调，可使对应任务的性能最高提升52.7个准确率点。其次，由于指令数据集可同时提升多个下游任务的性能，Cookbook通过算法学习如何混合来自不同模板的数据，以优化多任务性能。在标准多任务GPT4ALL评估套件上，使用Cookbook生成的数据集微调的Mistral-7B，与其他70亿参数的指令微调模型相比，平均准确率最高，且在8项任务中有3项表现最佳。最后，我们分析了Cookbook提升性能的时机和原因，并提出了一个指标，用以验证性能提升主要源于模型生成结果更符合模板规则。
英文摘要：
Fine-tuning large language models (LLMs) on instruction datasets is a common way to improve their generative capabilities. However, instruction datasets can be expensive and time-consuming to manually curate, and while LLM-generated data is less labor-intensive, it may violate user privacy agreements or terms of service of LLM providers. Therefore, we seek a way of constructing instruction datasets with samples that are not generated by humans or LLMs but still improve LLM generative capabilities. In this work, we introduce Cookbook, a framework that programmatically generates training data consisting of simple patterns over random tokens, resulting in a scalable, cost-effective approach that avoids legal and privacy issues. First, Cookbook uses a template -- a data generating Python function -- to produce training data that encourages the model to learn an explicit pattern-based rule that corresponds to a desired task. We find that fine-tuning on Cookbook-generated data is able to improve performance on its corresponding task by up to 52.7 accuracy points. Second, since instruction datasets improve performance on multiple downstream tasks simultaneously, Cookbook algorithmically learns how to mix data from various templates to optimize performance on multiple tasks. On the standard multi-task GPT4ALL evaluation suite, Mistral-7B fine-tuned using a Cookbook-generated dataset attains the best accuracy on average compared to other 7B parameter instruction-tuned models and is the best performing model on 3 out of 8 tasks. Finally, we analyze when and why Cookbook improves performance and present a metric that allows us to verify that the improvement is largely explained by the model's generations adhering better to template rules.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
