DualDE：双重提取知识图谱嵌入以实现更快更廉价的推理
DualDE: Dually Distilling Knowledge Graph Embedding for Faster and Cheaper Reasoning
摘要：
知识图谱嵌入（KGE）是KG推理的一种流行方法，通常首选训练维度较高的知识图谱嵌入，因为它们具有更好的推理能力。然而，高维KGE对存储和计算资源提出了巨大的挑战，并且不适合于资源有限或时间受限的应用，对于这些应用，需要更快和更便宜的推理。为了解决这个问题，我们提出了DualDE，一个知识蒸馏的方法来建立低维的学生KGE从预先训练的高维教师KGE。DualDE考虑了教师和学生之间的双重影响。在DualDE中，我们提出了一种软标签评价机制，自适应地为不同的三元组分配不同的软标签和硬标签权重，以及一种两阶段蒸馏方法，以提高学生对教师的接受度。我们的DualDE是通用的，足以适用于各种KGE。实验结果表明，该方法在保持高性能的前提下，将高维KGE的嵌入参数减少了7倍~ 15倍，推理速度提高了2倍~ 6倍。我们还通过烧蚀研究实验证明了我们的软标签评估机制和两阶段蒸馏方法的有效性。
英文摘要：
Knowledge Graph Embedding (KGE) is a popular method for KG reasoning and training KGEs with higher dimension are usually preferred since they have better reasoning capability. However, high-dimensional KGEs pose huge challenges to storage and computing resources and are not suitable for resource-limited or time-constrained applications, for which faster and cheaper reasoning is necessary. To address this problem, we propose DualDE, a knowledge distillation method to build low-dimensional student KGE from pre-trained high-dimensional teacher KGE. DualDE considers the dual-influence between the teacher and the student. In DualDE, we propose a soft label evaluation mechanism to adaptively assign different soft label and hard label weights to different triples, and a two-stage distillation approach to improve the student's acceptance of the teacher. Our DualDE is general enough to be applied to various KGEs. Experimental results show that our method can successfully reduce the embedding parameters of a high-dimensional KGE by 7 times - 15 times and increase the inference speed by 2 times - 6 times while retaining a high performance. We also experimentally prove the effectiveness of our soft label evaluation mechanism and two-stage distillation approach via ablation study.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
