The bilinear method is mainstream in Knowledge Graph Embedding (KGE), aiming to learn low-dimensional representations for entities and relations in Knowledge Graph (KG) and complete missing links. Most of the existing works are to find patterns between relationships and effectively model them to accomplish this task. Previous works have mainly discovered 6 important patterns like non-commutativity. Although some bilinear methods succeed in modeling these patterns, they neglect to handle 1-to-N, N-to-1, and N-to-N relations (or complex relations) concurrently, which hurts their expressiveness. To this end, we integrate scaling, the combination of translation and rotation that can solve complex relations and patterns, respectively, where scaling is a simplification of projection. Therefore, we propose a corresponding bilinear model Scaling Translation and Rotation (STaR) consisting of the above two parts. Besides, since translation cannot be incorporated into the bilinear model directly, we introduce translation matrix as the equivalent. Theoretical analysis proves that STaR is capable of modeling all patterns and handling complex relations simultaneously, and experiments demonstrate its effectiveness on commonly used benchmarks for link prediction.
嵌入矩阵是神经自然语言处理(NLP)模型中的关键组件,负责提供输入标记的数值表示。\footnote{In this paper words and subwords are referred to as \textit{tokens} and the term \textit{embedding} only refers to embeddings of inputs.}在本文中,我们分析了神经机器翻译(NMT)的背景下,这种矩阵的影响和效用。我们发现,从词嵌入中提取句法和语义信息,并使用随机嵌入运行NMT系统,并不像最初听起来那样具有破坏性。我们还展示了如何从经过充分训练的嵌入中只整合有限数量的特定于任务的知识,从而提高NMT系统的性能。我们的研究结果表明,在交换性能可以忽略不计的恶化,任何NMT模型可以运行部分随机嵌入。使用这种结构意味着最小的内存需求,因为不再需要存储大型嵌入表,这在工业和设备设置中是一个显着的增益。我们在将{English}翻译成{German}和{French}时评估了我们的嵌入,并实现了$5.3$ x压缩率。尽管有一个相当小的架构,我们的模型在某些情况下甚至能够超越最先进的基线。