arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2306.04751cs.CL
收藏

骆驼能走多远?探索开放资源上的指令微调现状

How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources

Allen Institute for AI(艾伦人工智能研究所);University of Washington(华盛顿大学)

Yizhong Wang, Hamish Ivison, Pradeep Dasigi, Jack Hessel, Tushar Khot, Khyathi Raghavi Chandu, David Wadden, Kelsey MacMillan, Noah A. Smith, Iz Beltagy, Hannaneh Hajishirzi

AI总结:

本文系统评估了基于12个开放指令数据集训练的6.7B至65B指令微调模型,提出最佳模型套件Tülu,并发现最佳模型平均仅达ChatGPT的87%和GPT-4的73%,揭示开放模型与专有模型的差距及系统性评估的必要性。

摘要:

在这项工作中,我们探索了在一系列开放指令遵循数据集上对语言模型进行指令微调的最新进展。尽管近期有声称开放模型可以与最先进的专有模型相媲美,但这些声称往往伴随着有限的评估,使得难以全面比较各模型并确定各种资源的效用。我们提供了一组规模从6.7B到65B参数的大规模指令微调模型,这些模型在12个指令数据集上进行了训练,数据集范围从人工整理(如OpenAssistant)到合成与蒸馏(如Alpaca),并通过一系列自动、基于模型和基于人类的指标,系统评估了它们在事实知识、推理、多语言能力、编码和开放式指令遵循能力方面的表现。我们进一步引入了Tülu,这是我们表现最佳的指令微调模型套件,它在高质量开放资源的组合上进行了微调。我们的实验表明,不同的指令微调数据集可以揭示或增强特定技能,而没有任何单一数据集(或组合)能在所有评估中提供最佳性能。有趣的是,我们发现基于模型和人类偏好的评估未能反映基于基准的评估所揭示的模型能力差异,这表明需要像本工作所执行的系统性评估类型。我们的评估显示,在任何给定评估中,最佳模型平均达到ChatGPT性能的87%,以及GPT-4性能的73%,这表明需要进一步投资于构建更好的基础模型和指令微调数据以缩小差距。我们在https://github.com/allenai/open-instruct上发布了我们的指令微调模型,包括一个完全微调的65B Tülu,以及我们的代码、数据和评估框架,以促进未来的研究。

英文摘要:

In this work we explore recent advances in instruction-tuning language models on a range of open instruction-following datasets. Despite recent claims that open models can be on par with state-of-the-art proprietary models, these claims are often accompanied by limited evaluation, making it difficult to compare models across the board and determine the utility of various resources. We provide a large set of instruction-tuned models from 6.7B to 65B parameters in size, trained on 12 instruction datasets ranging from manually curated (e.g., OpenAssistant) to synthetic and distilled (e.g., Alpaca) and systematically evaluate them on their factual knowledge, reasoning, multilinguality, coding, and open-ended instruction following abilities through a collection of automatic, model-based, and human-based metrics. We further introduce Tülu, our best performing instruction-tuned model suite finetuned on a combination of high-quality open resources. Our experiments show that different instruction-tuning datasets can uncover or enhance specific skills, while no single dataset (or combination) provides the best performance across all evaluations. Interestingly, we find that model and human preference-based evaluations fail to reflect differences in model capabilities exposed by benchmark-based evaluations, suggesting the need for the type of systemic evaluation performed in this work. Our evaluations show that the best model in any given evaluation reaches on average 87% of ChatGPT performance, and 73% of GPT-4 performance, suggesting that further investment in building better base models and instruction-tuning data is required to close the gap. We release our instruction-tuned models, including a fully finetuned 65B Tülu, along with our code, data, and evaluation framework at https://github.com/allenai/open-instruct to facilitate future research.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑