arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2203.09161cs.CLcs.AIcs.CVcs.LG
收藏

额外的指令值得使用多少个数据样本?

How Many Data Samples is an Additional Instruction Worth?

Arizona State University(亚利桑那州立大学)

Ravsehaj Singh Puri, Swaroop Mishra, Mihir Parmar, Chitta Baral

摘要:

最近引入的推理范式使非专家用户能够通过用自然语言定义新任务来利用NLP资源。指令调整模型的表现明显优于多任务学习模型(没有指令),但它们远不是最先进的特定任务模型。通过创建具有大量任务实例或模型中的架构变化的数据集来提高模型性能的常规方法对于非专家用户可能是不可行的。但是,它们可以编写替代指令来表示指令任务。增强教学有帮助吗?我们增加了一个子集的任务,在扩展版本的自然指令与额外的指令,并发现它显着提高模型的性能(高达35%),特别是在低数据制度。我们的研究结果表明,一个额外的指令可以相当于约200个数据样本平均跨任务。

英文摘要:

Recently introduced instruction-paradigm empowers non-expert users to leverage NLP resources by defining a new task in natural language. Instruction-tuned models have significantly outperformed multitask learning models (without instruction); however they are far from state-of-the-art task-specific models. Conventional approaches to improve model performance via creating datasets with large number of task instances or architectural changes in the model may not be feasible for non-expert users. However, they can write alternate instructions to represent an instruction task. Is Instruction-augmentation helpful? We augment a subset of tasks in the expanded version of NATURAL INSTRUCTIONS with additional instructions and find that it significantly improves model performance (up to 35%), especially in the low-data regime. Our results indicate that an additional instruction can be equivalent to ~200 data samples on average across tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑