arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2004.00584cs.DBcs.CL
收藏

使用预训练的语言模型进行深度实体匹配

Deep Entity Matching with Pre-Trained Language Models

Megagon Labs(Megagon实验室);University of Wisconsin Madison(威斯康星大学麦迪逊分校)

Yuliang Li, Jinfeng Li, Yoshihiko Suhara, AnHai Doan, Wang-Chiew Tan

摘要:

我们提出了Ditto,一个新的实体匹配系统的基础上预先训练的transformer为基础的语言模型。我们微调和铸造EM作为一个序列对分类问题,利用这样的模型与一个简单的架构。我们的实验表明,在大型文本语料库上预训练的语言模型(如BERT,DistilBERT或RoBERTa)的直接应用已经显着提高了匹配质量,并优于以前的最先进技术(SOTA),在基准数据集上的F1得分高达29%。我们还开发了三种优化技术,以进一步提高Ditto的匹配能力。Ditto允许通过突出显示在做出匹配决策时可能感兴趣的重要输入信息来注入领域知识。Ditto还总结了太长的字符串,以便只保留必要的信息并用于EM。最后,Ditto在文本到EM的数据增强上采用SOTA技术,用(困难的)示例来增强训练数据。通过这种方式,Ditto被迫“更努力”地学习,以提高模型的匹配能力。我们开发的优化进一步将Ditto的性能提高了9.8%。也许更令人惊讶的是,我们确定Ditto可以用最多一半的标记数据实现之前的SOTA结果。最后,我们证明了Ditto在现实世界的大规模EM任务上的有效性。在匹配由789K和412K记录组成的两个公司数据集时,Ditto获得了96.5%的F1高分。

英文摘要:

We present Ditto, a novel entity matching system based on pre-trained Transformer-based language models. We fine-tune and cast EM as a sequence-pair classification problem to leverage such models with a simple architecture. Our experiments show that a straightforward application of language models such as BERT, DistilBERT, or RoBERTa pre-trained on large text corpora already significantly improves the matching quality and outperforms previous state-of-the-art (SOTA), by up to 29% of F1 score on benchmark datasets. We also developed three optimization techniques to further improve Ditto's matching capability. Ditto allows domain knowledge to be injected by highlighting important pieces of input information that may be of interest when making matching decisions. Ditto also summarizes strings that are too long so that only the essential information is retained and used for EM. Finally, Ditto adapts a SOTA technique on data augmentation for text to EM to augment the training data with (difficult) examples. This way, Ditto is forced to learn "harder" to improve the model's matching capability. The optimizations we developed further boost the performance of Ditto by up to 9.8%. Perhaps more surprisingly, we establish that Ditto can achieve the previous SOTA results with at most half the number of labeled data. Finally, we demonstrate Ditto's effectiveness on a real-world large-scale EM task. On matching two company datasets consisting of 789K and 412K records, Ditto achieves a high F1 score of 96.5%.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑