arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2109.02772cs.AI
收藏

基于预训练语言模型的Few-Shot知识探测的实证研究

An Empirical Study on Few-shot Knowledge Probing for Pretrained Language Models

MIT(麻省理工学院);New York University(纽约大学)

Tianxing He, Kyunghyun Cho, James Glass

摘要:

1-hop关系的基于语义的知识探测已被用于测量有多少世界知识存储在预训练的语言模型中。现有的工作使用大量的数据来调整提示以获得更好的性能。在这项工作中,我们比较了各种方法下的Few-Shot知识探测设置,其中只有少数(例如,10或20)的示例三元组可用。此外,我们创建了一个名为TREx-2 p的新数据集,其中包含2跳关系。我们报告说,Few-Shot的例子可以大大提高探测性能的1跳和2跳的关系。特别是,我们发现,一个简单而有效的方法微调模型中的偏置向量优于现有的工程方法。我们的数据集和代码可以在\url{https://github.com/cloudygoose/fewshot_lama}上找到。

英文摘要:

Prompt-based knowledge probing for 1-hop relations has been used to measure how much world knowledge is stored in pretrained language models. Existing work uses considerable amounts of data to tune the prompts for better performance. In this work, we compare a variety of approaches under a few-shot knowledge probing setting, where only a small number (e.g., 10 or 20) of example triples are available. In addition, we create a new dataset named TREx-2p, which contains 2-hop relations. We report that few-shot examples can strongly boost the probing performance for both 1-hop and 2-hop relations. In particular, we find that a simple-yet-effective approach of finetuning the bias vectors in the model outperforms existing prompt-engineering methods. Our dataset and code are available at \url{https://github.com/cloudygoose/fewshot_lama}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑