The ability of knowledge graphs to represent complex relationships at scale has led to their adoption for various needs including knowledge representation, question-answering, and recommendation systems. Knowledge graphs are often incomplete in the information they represent, necessitating the need for knowledge graph completion tasks. Pre-trained and fine-tuned language models have shown promise in these tasks although these models ignore the intrinsic information encoded in the knowledge graph, namely the entity and relation types. In this work, we propose the Knowledge Graph Language Model (KGLM) architecture, where we introduce a new entity/relation embedding layer that learns to differentiate distinctive entity and relation types, therefore allowing the model to learn the structure of the knowledge graph. In this work, we show that further pre-training the language models with this additional embedding layer using the triples extracted from the knowledge graph, followed by the standard fine-tuning phase sets a new state-of-the-art performance for the link prediction task on the benchmark datasets.
Leveraging Graph Structure in Seq2Seq Models for Knowledge Graph Link Prediction
Analytic Computing, KI, University of Stuttgart, Stuttgart, Germany(斯图加特大学分析计算研究所);Bosch Center for Artificial Intelligence, Stuttgart, Germany(博世人工智能中心);WAIS, University of Southampton, United Kingdom(南安普顿大学WAIS)
Wiki-TabNER: Integrating Named Entity Recognition into Wikipedia Tables
Ludwig-Maximilians-Universität Munich Germany(慕尼黑路德维希-马克西米利安大学);Siemens AG Munich Germany(西门子股份公司);Technical University of Munich Munich Germany(慕尼黑技术大学)
PageRank without hyperlinks: Structural re-ranking using links induced by language models
受网页搜索的PageRank和HITS(hubs and authorities)算法的启发,我们提出了一种结构化的重新排序方法,用于ad hoc信息检索:我们通过利用它们之间的非对称关系对初始检索集中的文档进行重新排序。具体来说,我们考虑生成链接,这表明从一个文档诱导的语言模型分配给另一个文本的概率很高,在这样做的时候,我们要注意防止对长文档的偏见。我们研究了一些重新排名的标准的基础上的措施,在生成链接形成的图形的中心性,并表明,集成中心性到标准的基于语言模型的检索是非常有效的,在提高精度的顶级。
Integrating Neurosymbolic AI in Advanced Air Mobility: A Comprehensive Survey
Department of Information Systems, University of Maryland, Baltimore County(信息系统系,马里兰大学巴尔的摩分校);Department of Mechanical and Aerospace Engineering, The George Washington University(机械与航空航天工程系,乔治华盛顿大学);Department of Mechanical Engineering, Baylor University(机械工程系,贝勒大学)
LINKED: Eliciting, Filtering and Integrating Knowledge in Large Language Model for Commonsense Reasoning
School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所)