arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2010.11967cs.CLcs.AIcs.LG
收藏

语言模型是开放的知识图谱

Language Models are Open Knowledge Graphs

UC Berkeley(加州大学伯克利分校);Tsinghua University(清华大学)

Chenguang Wang, Xiao Liu, Dawn Song

摘要:

本文展示了如何从预训练的语言模型(例如,BERT,GPT-2/3),无需人工监督。流行的知识库(例如,Wikidata,NELL)以监督或半监督的方式构建,需要人类创造知识。最近的深度语言模型通过预训练自动从大规模语料库中获取知识。存储的知识使得语言模型能够改进下游NLP任务,例如,回答问题,写代码和文章。在本文中,我们提出了一种无监督的方法来铸造知识包含在语言模型到KG。我们表明,KG构建了一个单一的前向通过的预训练语言模型(没有微调)的语料库。我们通过与人类创建的两个KG(Wikidata,TAC KBP)进行比较来证明构建的KG的质量。我们的幼儿园还提供开放的事实知识,这是在现有的幼儿园新的。我们的代码和KG将公开提供。

英文摘要:

This paper shows how to construct knowledge graphs (KGs) from pre-trained language models (e.g., BERT, GPT-2/3), without human supervision. Popular KGs (e.g, Wikidata, NELL) are built in either a supervised or semi-supervised manner, requiring humans to create knowledge. Recent deep language models automatically acquire knowledge from large-scale corpora via pre-training. The stored knowledge has enabled the language models to improve downstream NLP tasks, e.g., answering questions, and writing code and articles. In this paper, we propose an unsupervised method to cast the knowledge contained within language models into KGs. We show that KGs are constructed with a single forward pass of the pre-trained language models (without fine-tuning) over the corpora. We demonstrate the quality of the constructed KGs by comparing to two KGs (Wikidata, TAC KBP) created by humans. Our KGs also provide open factual knowledge that is new in the existing KGs. Our code and KGs will be made publicly available.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑