arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2206.07558cs.CLcs.AI
收藏

实体和关系抽取中的上下文化和泛化

Contextualization and Generalization in Entity and Relation Extraction

Sorbonne Université(索邦大学);Aix-Marseille Université(艾克斯-马赛大学);La Rochelle Université(拉罗谢尔大学);École Polytechnique Fédérale de Lausanne(洛桑联邦理工学院)

Bruno Taillé

摘要:

在过去的十年中,神经网络在自然语言处理(NLP)中变得突出,特别是它们从大型未标记语料库中学习相关单词表示的能力。然后,这些词嵌入可以在监督训练阶段被转移和微调以用于不同的最终应用。最近,在2018年,整个预训练的语言模型的转移和它们的上下文能力的保留使得几乎所有NLP基准都达到了前所未有的性能,有时甚至超过了人类基线。然而,当模型达到如此令人印象深刻的分数时,它们的理解能力仍然显得肤浅,这揭示了基准的局限性,无法提供对其性能因素的有用见解,并准确测量理解能力。在这篇论文中,我们研究了两个重要的信息抽取任务:命名实体识别(NER)和关系抽取(RE)中,最先进的模型在训练过程中对未知事实的泛化行为。实际上,传统的基准测试在用于训练和评估模型的提及和关系之间存在重要的词汇重叠,而信息抽取的主要兴趣是提取先前未知的信息。我们提出了实证研究,以分离性能的基础上提到和关系重叠的训练集,并发现预训练的语言模型主要是有益的,以检测看不见的提及,特别是域外。虽然这使它们适合于真实的用例,但可见和不可见的提及之间仍然存在性能差距,这会损害对新事实的概括。特别是,即使是最先进的ERE模型也依赖于浅层保留启发式,其预测更多地基于参数表面形式而不是上下文。

英文摘要:

During the past decade, neural networks have become prominent in Natural Language Processing (NLP), notably for their capacity to learn relevant word representations from large unlabeled corpora. These word embeddings can then be transferred and finetuned for diverse end applications during a supervised training phase. More recently, in 2018, the transfer of entire pretrained Language Models and the preservation of their contextualization capacities enabled to reach unprecedented performance on virtually every NLP benchmark, sometimes even outperforming human baselines. However, as models reach such impressive scores, their comprehension abilities still appear as shallow, which reveal limitations of benchmarks to provide useful insights on their factors of performance and to accurately measure understanding capabilities. In this thesis, we study the behaviour of state-of-the-art models regarding generalization to facts unseen during training in two important Information Extraction tasks: Named Entity Recognition (NER) and Relation Extraction (RE). Indeed, traditional benchmarks present important lexical overlap between mentions and relations used for training and evaluating models, whereas the main interest of Information Extraction is to extract previously unknown information. We propose empirical studies to separate performance based on mention and relation overlap with the training set and find that pretrained Language Models are mainly beneficial to detect unseen mentions, in particular out-of-domain. While this makes them suited for real use cases, there is still a gap in performance between seen and unseen mentions that hurts generalization to new facts. In particular, even state-of-the-art ERE models rely on a shallow retention heuristic, basing their prediction more on arguments surface forms than context.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑