arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2102.09681cs.CLcs.IR
收藏

WebRED:Web上关系抽取的有效预训练和微调

WebRED: Effective Pretraining And Finetuning For Relation Extraction On The Web

Google Brain(谷歌大脑);Google(谷歌公司)

Robert Ormandi, Mohammad Saleh, Erin Winter, Vinay Rao

摘要:

关系抽取用于填充对许多应用程序很重要的知识库。用于训练关系抽取模型的现有数据集要么由于远程监督而遭受噪声标签,要么限于某些域,要么太小而无法训练高容量模型。这限制了关系抽取的下游应用。因此,我们介绍:WebRED(Web关系抽取数据集),一个强监督的人类注释数据集,用于从万维网上发现的各种文本中提取关系,由约110 K个示例组成。我们还描述了我们用来收集~ 200 M示例作为此任务的预训练数据的方法。我们表明,将大型弱监督数据集上的预训练与小型强监督数据集上的微调相结合,可以获得更好的关系抽取性能。我们为这个新的数据集提供了基线,并提出了一个案例,人类注释的重要性,提高性能的关系抽取从网络上找到的文本。

英文摘要:

Relation extraction is used to populate knowledge bases that are important to many applications. Prior datasets used to train relation extraction models either suffer from noisy labels due to distant supervision, are limited to certain domains or are too small to train high-capacity models. This constrains downstream applications of relation extraction. We therefore introduce: WebRED (Web Relation Extraction Dataset), a strongly-supervised human annotated dataset for extracting relationships from a variety of text found on the World Wide Web, consisting of ~110K examples. We also describe the methods we used to collect ~200M examples as pre-training data for this task. We show that combining pre-training on a large weakly supervised dataset with fine-tuning on a small strongly-supervised dataset leads to better relation extraction performance. We provide baselines for this new dataset and present a case for the importance of human annotation in improving the performance of relation extraction from text found on the web.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑