arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1911.03042cs.CL
收藏

自然语言处理中的神经图嵌入方法

Neural Graph Embedding Methods for Natural Language Processing

Indian Institute of Science(印度科学学院)

Shikhar Vashishth

摘要:

知识图谱是图中事实的结构化表示,其中节点表示实体,边表示它们之间的关系。最近的研究导致了几个大型幼儿园的发展。然而,它们都倾向于稀疏,每个实体的事实很少。在论文的第一部分,我们提出了两种解决方案来缓解这个问题:(1)KG规范化,即,识别和合并KG中的重复实体,(2)关系抽取,其涉及自动化从非结构化文本中抽取实体之间的语义关系的过程。像CNN和RNN这样的传统神经网络被限制为处理欧几里得数据。然而,图在自然语言处理(NLP)中是突出的。最近,图卷积网络(GCN)已被提出来解决这个缺点,并已成功地应用于几个问题。在论文的第二部分,我们利用GCN的文档时间戳问题和学习词嵌入使用一个词的依赖上下文,而不是顺序上下文。在论文的第三部分,我们解决了现有GCN模型的两个局限性,即,(1)标准的邻域聚合方案对可以影响目标节点的表示的节点的数量没有限制。这导致了枢纽节点的噪声表示,其在几跳中覆盖了几乎整个图。(2)现有的大多数GCN模型仅限于处理无向图。然而,一个更普遍和普遍的一类图是关系图,其中每个边都有一个标签和方向与之相关联。现有的方法来处理这样的图遭受过参数化,并仅限于学习表示的节点。

英文摘要:

Knowledge graphs are structured representations of facts in a graph, where nodes represent entities and edges represent relationships between them. Recent research has resulted in the development of several large KGs. However, all of them tend to be sparse with very few facts per entity. In the first part of the thesis, we propose two solutions to alleviate this problem: (1) KG Canonicalization, i.e., identifying and merging duplicate entities in a KG, (2) Relation Extraction which involves automating the process of extracting semantic relationships between entities from unstructured text. Traditional Neural Networks like CNNs and RNNs are constrained to handle Euclidean data. However, graphs in Natural Language Processing (NLP) are prominent. Recently, Graph Convolutional Networks (GCNs) have been proposed to address this shortcoming and have been successfully applied for several problems. In the second part of the thesis, we utilize GCNs for Document Timestamping problem and for learning word embeddings using dependency context of a word instead of sequential context. In this third part of the thesis, we address two limitations of existing GCN models, i.e., (1) The standard neighborhood aggregation scheme puts no constraints on the number of nodes that can influence the representation of a target node. This leads to a noisy representation of hub-nodes which coves almost the entire graph in a few hops. (2) Most of the existing GCN models are limited to handle undirected graphs. However, a more general and pervasive class of graphs are relational graphs where each edge has a label and direction associated with it. Existing approaches to handle such graphs suffer from over-parameterization and are restricted to learning representation of nodes only.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑