arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2210.10693cs.CLcs.AIcs.LG
收藏

有限数据情境下演示学习的鲁棒性

Robustness of Demonstration-based Learning Under Limited Data Scenario

Shanghai Jiao Tong University(上海交通大学);Georgia Institute of Technology(佐治亚理工学院);Adobe Research(Adobe研究院);Stanford University(斯坦福大学)

Hongxin Zhang, Yanzhe Zhang, Ruiyi Zhang, Diyi Yang

摘要:

基于演示的学习在有限数据场景下激发预先训练好的语言模型的能力方面显示出巨大的潜力。通过一些演示简单地增加输入可以显著提高Few-Shot NER的性能。然而,为什么这样的演示是有益的学习过程仍然不清楚,因为没有明确的示威和预测之间的一致性。本文通过逐步去除标准样本中直观有用的信息来设计病态样本,深入研究了基于样本的序列标注的鲁棒性,结果表明:(1)由随机标记组成的样本仍然使模型成为更好的Few-Shot学习器;(2)随机样本的长度和随机标记的相关性是影响性能的主要因素;(3)演示增加了模型预测对捕获的表面模式的置信度。我们已在https://github.com/SALT-NLP/RobustDemo上公开发布了我们的代码。

英文摘要:

Demonstration-based learning has shown great potential in stimulating pretrained language models' ability under limited data scenario. Simply augmenting the input with some demonstrations can significantly improve performance on few-shot NER. However, why such demonstrations are beneficial for the learning process remains unclear since there is no explicit alignment between the demonstrations and the predictions. In this paper, we design pathological demonstrations by gradually removing intuitively useful information from the standard ones to take a deep dive of the robustness of demonstration-based sequence labeling and show that (1) demonstrations composed of random tokens still make the model a better few-shot learner; (2) the length of random demonstrations and the relevance of random tokens are the main factors affecting the performance; (3) demonstrations increase the confidence of model predictions on captured superficial patterns. We have publicly released our code at https://github.com/SALT-NLP/RobustDemo.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑