arXiv 2401.067512024-01cs.CLcs.AIcs.LG

Easy Training Data对Hard Tasks的惊人有效性

The Unreasonable Effectiveness of Easy Training Data for Hard Tasks

Allen Institute for AI（艾伦人工智能研究所）；UNC Chapel Hill（北卡罗来纳大学教堂山分校）

Peter Hase, Mohit Bansal, Peter Clark, Sarah Wiegreffe

备注：: ACL 2024. 23 pages, 20 figures

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对可扩展监督问题，本文发现预训练语言模型能从简单数据有效泛化到困难数据，表现媲美在困难数据上微调的模型，且收集简单数据进行微调往往更具性价比。

摘要：

当hard training data本身难以正确标注时，如何训练模型在hard test data上表现良好？这个问题被称为scalable oversight problem，随着语言模型不断改进而受到关注。本文得出惊人结论：当前pretrained language models通常能从easy到hard data较好泛化，甚至与在hard data上finetune的oracle models表现相当。我们使用in-context learning、linear classifier heads和QLoRA等简单finetuning方法，针对七种datapoint hardness度量展示了这种easy-to-hard generalization，包括六种经验性多样化的人类难度度量（如grade level）和一种基于模型的度量（loss-based）。此外，我们表明即使最关心模型在hard data上的表现，收集easy data而非hard data进行finetuning可能更好，因为hard data通常噪声更大且收集成本更高。我们的实验使用了规模达70b的open models和四个公开的question-answering datasets，问题难度从3rd grade science questions到college level STEM questions和general-knowledge trivia。我们得出结论：LMs在所研究任务中的easy-to-hard generalization惊人地强。代码见：https://github.com/allenai/easy-to-hard-generalization

英文摘要：

How can we train models to perform well on hard test data when hard training data is by definition difficult to label correctly? This question has been termed the scalable oversight problem and has drawn increasing attention as language models have continually improved. In this paper, we present the surprising conclusion that current pretrained language models often generalize relatively well from easy to hard data, even performing as well as oracle models finetuned on hard data. We demonstrate this kind of easy-to-hard generalization using simple finetuning methods like in-context learning, linear classifier heads, and QLoRA for seven different measures of datapoint hardness, including six empirically diverse human hardness measures (like grade level) and one model-based measure (loss-based). Furthermore, we show that even if one cares most about model performance on hard data, it can be better to collect easy data rather than hard data for finetuning, since hard data is generally noisier and costlier to collect. Our experiments use open models up to 70b in size and four publicly available question-answering datasets with questions ranging in difficulty from 3rd grade science questions to college level STEM questions and general-knowledge trivia. We conclude that easy-to-hard generalization in LMs is surprisingly strong for the tasks studied. Our code is available at: https://github.com/allenai/easy-to-hard-generalization

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

AI总结：

摘要：

英文摘要：

相关论文