arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.06097cs.AIcs.LG
收藏

TWIG:通过模拟KGE模型实现预超参数优化与跨图泛化

TWIG: Towards pre-hoc Hyperparameter Optimisation and Cross-Graph Generalisation via Simulated KGE Models

Trinity College Dublin(都柏林圣三一大学)

Jeffrey Sardina, John D. Kelleher, Declan O'Sullivan

AI总结:

TWIG通过仅用2590个参数模拟ComplEx-N3模型在UMLS上的结果,证明KGE仅学习结构模式,并提出结构泛化假设,实现跨图链接预测。

摘要:

在本文中,我们引入了TWIG(拓扑加权智能生成),一种新颖的、无嵌入的范式,用于模拟KGE(知识图嵌入)的输出,且仅使用极小部分的参数。TWIG从由图的拓扑特征组成的输入中学习权重,无需对实体或边的潜在表示进行编码。我们在UMLS数据集上的实验表明,一个单一的TWIG神经网络能够在所有超参数配置下几乎精确地预测最先进的ComplEx-N3 KGE模型的结果。为此,它总共使用了2590个可学习参数,但准确预测了1215种不同超参数组合的结果,这些组合的总参数成本为29,322,000个。基于这些结果,我们提出两个主张:1)KGE并不学习潜在语义,而只是学习结构模式的潜在表示;2)KGE中的超参数选择是KGE模型和图结构的确定性函数。我们进一步假设,由于TWIG可以在没有嵌入的情况下模拟KGE,节点和边嵌入并非学习准确预测知识图谱中新事实所必需的。最后,我们将所有发现归纳在“结构泛化假设”的框架下,该假设表明,“细枝状”的无嵌入/基于数据结构的学习方法可以使单个神经网络模拟KGE性能,并可能解决跨多个来自不同领域且具有不同语义的知识图谱的链接预测任务。

英文摘要:

In this paper we introduce TWIG (Topologically-Weighted Intelligence Generation), a novel, embedding-free paradigm for simulating the output of KGEs that uses a tiny fraction of the parameters. TWIG learns weights from inputs that consist of topological features of the graph data, with no coding for latent representations of entities or edges. Our experiments on the UMLS dataset show that a single TWIG neural network can predict the results of state-of-the-art ComplEx-N3 KGE model nearly exactly on across all hyperparameter configurations. To do this it uses a total of 2590 learnable parameters, but accurately predicts the results of 1215 different hyperparameter combinations with a combined cost of 29,322,000 parameters. Based on these results, we make two claims: 1) that KGEs do not learn latent semantics, but only latent representations of structural patterns; 2) that hyperparameter choice in KGEs is a deterministic function of the KGE model and graph structure. We further hypothesise that, as TWIG can simulate KGEs without embeddings, that node and edge embeddings are not needed to learn to accurately predict new facts in KGs. Finally, we formulate all of our findings under the umbrella of the ``Structural Generalisation Hypothesis", which suggests that ``twiggy" embedding-free / data-structure-based learning methods can allow a single neural network to simulate KGE performance, and perhaps solve the Link Prediction task, across many KGs from diverse domains and with different semantics.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑