多偏差并行建模提高QA泛化能力
Improving QA Generalization by Concurrent Modeling of Multiple Biases
摘要:
现有的NLP数据集包含各种偏见,模型可以很容易地利用这些偏见来实现相应评估集的高性能。然而,专注于特定于数据集的偏见限制了他们从更一般的数据模式中学习有关任务的更一般化知识的能力。在本文中,我们研究了去偏方法对提高泛化能力的影响,并提出了一个通用框架,通过对训练数据中的多个偏差进行并发建模,来提高域内和域外数据集的性能。我们的框架根据每个示例所包含的偏差以及这些偏差在训练数据中的强度来对每个示例进行加权。然后,它在训练目标中使用这些权重,以便模型更少地依赖于具有高偏差权重的示例。我们广泛评估我们的框架提取问答与训练数据从不同领域的不同强度的多个偏见。我们在两种不同的设置中进行评估,其中模型同时在单个域或多个域上进行训练,并与最先进的去偏置方法相比,在两种设置中显示其有效性。
英文摘要:
Existing NLP datasets contain various biases that models can easily exploit to achieve high performances on the corresponding evaluation sets. However, focusing on dataset-specific biases limits their ability to learn more generalizable knowledge about the task from more general data patterns. In this paper, we investigate the impact of debiasing methods for improving generalization and propose a general framework for improving the performance on both in-domain and out-of-domain datasets by concurrent modeling of multiple biases in the training data. Our framework weights each example based on the biases it contains and the strength of those biases in the training data. It then uses these weights in the training objective so that the model relies less on examples with high bias weights. We extensively evaluate our framework on extractive question answering with training data from various domains with multiple biases of different strengths. We perform the evaluations in two different settings, in which the model is trained on a single domain or multiple domains simultaneously, and show its effectiveness in both settings compared to state-of-the-art debiasing methods.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。