arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2210.01979cs.CLcs.AI
收藏

GAPX:广义自回归释义识别

GAPX: Generalized Autoregressive Paraphrase-Identification X

Cornell University(康奈尔大学);Meta AI

Yifei Zhou, Renyu Li, Hayden Housen, Ser-Nam Lim

摘要:

释义识别是自然语言处理中的一项基本任务。虽然在该领域已经取得了很大的进展,许多国家的最先进的模型的性能往往受到分布偏移在推理时间。我们验证,这种性能下降的一个主要来源来自负面的例子引入的偏见。为了克服这些偏差,我们在本文中提出了训练两个单独的模型,一个只利用正对,另一个负对。这使我们能够选择如何利用负模型,为此,我们引入了一个基于困惑度的分布外度量,我们表明它可以有效地自动确定在推理过程中应该给予多少权重。我们支持我们的研究结果与强有力的实证结果。

英文摘要:

Paraphrase Identification is a fundamental task in Natural Language Processing. While much progress has been made in the field, the performance of many state-of-the-art models often suffer from distribution shift during inference time. We verify that a major source of this performance drop comes from biases introduced by negative examples. To overcome these biases, we propose in this paper to train two separate models, one that only utilizes the positive pairs and the other the negative pairs. This enables us the option of deciding how much to utilize the negative model, for which we introduce a perplexity based out-of-distribution metric that we show can effectively and automatically determine how much weight it should be given during inference. We support our findings with strong empirical results.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑