arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2211.01430cs.CL
收藏

向量空间中的层级结构:定向词嵌入与图嵌入

Hierarchies over Vector Space: Orienting Word and Graph Embeddings

Stony Brook University(石溪大学)

Xingzhi Guo, Steven Skiena

AI总结:

本文提出一种从向量嵌入中构建有向根树的数据结构,通过按实体影响力降序插入节点来捕获层级方向,在超词关系、LCA发现和维基链接恢复任务上显著优于基线。

摘要:

词嵌入和图嵌入在深度学习应用中被广泛使用。我们提出了一种数据结构,能够从无序的平坦嵌入空间中捕获固有的层级属性,特别是实体对之间的方向感。受“分布泛化性”概念的启发,我们的算法通过按实体影响力(如词频)降序插入节点来构建一个有向根树(树状结构),将每个实体指向其最近的、影响力更大的节点作为父节点。我们在三个任务上评估了所得树结构的性能:上位词关系发现、词间最低共同祖先(LCA)发现以及维基百科页面链接恢复。我们在五种语言上分别实现了上位词和LCA发现的平均准确率8.98%和2.70%,在有向维基页面链接恢复上实现了62.76%的准确率,均显著高于基线。最后,我们研究了插入顺序、影响力/相似度权衡以及各种影响力来源对父节点选择优化的影响。

英文摘要:

Word and graph embeddings are widely used in deep learning applications. We present a data structure that captures inherent hierarchical properties from an unordered flat embedding space, particularly a sense of direction between pairs of entities. Inspired by the notion of \textit{distributional generality}, our algorithm constructs an arborescence (a directed rooted tree) by inserting nodes in descending order of entity power (e.g., word frequency), pointing each entity to the closest more powerful node as its parent. We evaluate the performance of the resulting tree structures on three tasks: hypernym relation discovery, least-common-ancestor (LCA) discovery among words, and Wikipedia page link recovery. We achieve average 8.98\% and 2.70\% for hypernym and LCA discovery across five languages and 62.76\% accuracy on directed Wiki-page link recovery, with both substantially above baselines. Finally, we investigate the effect of insertion order, the power/similarity trade-off and various power sources to optimize parent selection.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑