arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2110.07736cs.CL
收藏

识别和消除伪相关性以提高NLP模型的鲁棒性

Identifying and Mitigating Spurious Correlations for Improving Robustness in NLP Models

University of Virginia(弗吉尼亚大学);Georgia Institute of Technology(佐治亚理工学院);Google Research(谷歌研究院)

Tianlu Wang, Rohit Sridhar, Diyi Yang, Xuezhi Wang

摘要:

最近,NLP模型在各种任务中取得了显着的进展;然而,它们也因不健壮而受到批评。许多鲁棒性问题可以归因于利用虚假相关性的模型,或者训练数据和任务标签之间的捷径。大多数现有的工作通过人类先验或错误分析确定了一组有限的特定于任务的捷径,这需要大量的专业知识和努力。在本文中,我们的目标是自动识别大规模NLP模型中的这种虚假相关性。我们首先利用现有的可解释性方法从输入文本中提取显著影响模型决策过程的标记。然后,我们通过分析跨多个语料库的模型预测来区分“真正的”令牌和“虚假的”令牌,并通过知识感知扰动进一步验证它们。我们表明,我们提出的方法可以有效地和高效地识别一组可扩展的“捷径”,并减轻这些导致在多个应用程序中更强大的模型。

英文摘要:

Recently, NLP models have achieved remarkable progress across a variety of tasks; however, they have also been criticized for being not robust. Many robustness problems can be attributed to models exploiting spurious correlations, or shortcuts between the training data and the task labels. Most existing work identifies a limited set of task-specific shortcuts via human priors or error analyses, which requires extensive expertise and efforts. In this paper, we aim to automatically identify such spurious correlations in NLP models at scale. We first leverage existing interpretability methods to extract tokens that significantly affect model's decision process from the input text. We then distinguish "genuine" tokens and "spurious" tokens by analyzing model predictions across multiple corpora and further verify them through knowledge-aware perturbations. We show that our proposed method can effectively and efficiently identify a scalable set of "shortcuts", and mitigating these leads to more robust models in multiple applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑