arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2206.09259cs.CL
收藏

语言模型能捕获图语义吗?从图到语言模型及其逆

Can Language Models Capture Graph Semantics? From Graphs to Language Model and Vice-Versa

BITS Pilani(比尔拉科学技术学院皮拉尼分校);AI Institute, University of South Carolina(南卡罗来纳大学人工智能研究所);University of South Carolina(南卡罗来纳大学)

Tarun Garg, Kaushik Roy, Amit Sheth

摘要:

知识图谱是一种很好的资源,可以用来捕获实体和实体之间关系方面的语义知识。然而,当前的深度学习模型将分布式表示或向量作为输入。因此,图被压缩在矢量化表示中。我们进行了一项研究,以检查深度学习模型是否可以压缩一个图,然后输出相同的图,其中大部分语义都完好无损。我们的实验表明,Transformer模型不能表达输入知识图谱的全部语义。我们发现,这是由于知识图谱中包含的有向、关系和基于类型的信息与Transformer注意矩阵的完全连接的令牌-令牌无向图形解释之间的差异。

英文摘要:

Knowledge Graphs are a great resource to capture semantic knowledge in terms of entities and relationships between the entities. However, current deep learning models takes as input distributed representations or vectors. Thus, the graph is compressed in a vectorized representation. We conduct a study to examine if the deep learning model can compress a graph and then output the same graph with most of the semantics intact. Our experiments show that Transformer models are not able to express the full semantics of the input knowledge graph. We find that this is due to the disparity between the directed, relationship and type based information contained in a Knowledge Graph and the fully connected token-token undirected graphical interpretation of the Transformer Attention matrix.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑