arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2205.09295cs.CL
收藏

基于数据库的模型是无线索的吗?

Are Prompt-based Models Clueless?

Tohoku University(东北大学);RIKEN AIP(理化学研究所先进智能项目中心);LegalForce Research(LegalForce研究院)

Pride Kavumba, Ryo Takahashi, Yusuke Oda

摘要:

用特定于任务的头部对大型预训练语言模型进行微调,在许多自然语言理解基准测试中推进了最先进的技术。然而,具有特定任务头部的模型需要大量的训练数据,这使得它们容易学习和利用特定于特定于任务的表面线索,而这些线索不会推广到其他数据集。通过重用语言模型头部并格式化任务输入以匹配预训练目标,训练减少了数据需求。因此,预计基于Few-Shot的模型不会利用表面线索。本文提出了一个实证检验是否Few-Shot基于模型也利用表面线索。分析基于Few-Shot的MNLI、SNLI、HANS和COPA模型,发现基于Bottom的模型也利用了表面线索。虽然模型在具有表面线索的实例上表现良好,但在没有表面线索的实例上,它们往往表现不佳或仅略微优于随机准确性。

英文摘要:

Finetuning large pre-trained language models with a task-specific head has advanced the state-of-the-art on many natural language understanding benchmarks. However, models with a task-specific head require a lot of training data, making them susceptible to learning and exploiting dataset-specific superficial cues that do not generalize to other datasets. Prompting has reduced the data requirement by reusing the language model head and formatting the task input to match the pre-training objective. Therefore, it is expected that few-shot prompt-based models do not exploit superficial cues. This paper presents an empirical examination of whether few-shot prompt-based models also exploit superficial cues. Analyzing few-shot prompt-based models on MNLI, SNLI, HANS, and COPA has revealed that prompt-based models also exploit superficial cues. While the models perform well on instances with superficial cues, they often underperform or only marginally outperform random accuracy on instances without superficial cues.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑