arXiv 2009.075032020-09cs.CLcs.AIcs.LG

联合实体和关系抽取中Seq2Seq模型的暴露偏差最小化

Minimize Exposure Bias of Seq2Seq Models in Joint Entity and Relation Extraction

University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Kyoto University（京都大学）；Hunan Normal University（湖南师范大学）；Waseda University（早稻田大学）

Ranran Haoran Zhang, Qianying Liu, Aysa Xuemo Fan, Heng Ji, Daojian Zeng, Fei Cheng, Daisuke Kawahara, Sadao Kurohashi

备注：: EMNLP 2020 Findings

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

摘要：

实体和关系抽取联合抽取的目的是直接从纯文本中抽取关系三元组。先前的工作利用序列到序列（Seq2Seq）模型来生成三联体序列。然而，Seq2Seq对无序三元组强制执行不必要的顺序，并且涉及与错误累积相关联的大解码长度。这些引入暴露偏差，这可能导致模型过拟合频繁的标签组合，从而恶化泛化。我们提出了一种新的序列到无序多树（Seq2UMTree）模型，通过将解码长度限制为三个三元组内的三个，并删除三元组之间的顺序，以尽量减少曝光偏差的影响。我们在两个数据集DuIE和NYT上评估了我们的模型，并系统地研究了暴露偏倚如何改变Seq2Seq模型的性能。实验表明，最先进的Seq2Seq模型对两个数据集都过拟合，而Seq2UMTree显示出更好的泛化能力。我们的代码可在https://github.com/WindChimeRan/OpenJERE上获得。

英文摘要：

Joint entity and relation extraction aims to extract relation triplets from plain text directly. Prior work leverages Sequence-to-Sequence (Seq2Seq) models for triplet sequence generation. However, Seq2Seq enforces an unnecessary order on the unordered triplets and involves a large decoding length associated with error accumulation. These introduce exposure bias, which may cause the models overfit to the frequent label combination, thus deteriorating the generalization. We propose a novel Sequence-to-Unordered-Multi-Tree (Seq2UMTree) model to minimize the effects of exposure bias by limiting the decoding length to three within a triplet and removing the order among triplets. We evaluate our model on two datasets, DuIE and NYT, and systematically study how exposure bias alters the performance of Seq2Seq models. Experiments show that the state-of-the-art Seq2Seq model overfits to both datasets while Seq2UMTree shows significantly better generalization. Our code is available at https://github.com/WindChimeRan/OpenJERE .

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

摘要：

英文摘要：

相关论文