arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.00936cs.CLcs.AI
收藏

大语言模型增强的知识表示学习:综述

Large Language Model Enhanced Knowledge Representation Learning: A Survey

College of Intelligence and Computing, Tianjin University(天津大学智能与计算学院);College of Design and Innovation, Tongji University(同济大学设计与创新学院);Faculty of Science and Technology, University of Macao(澳门大学科技学院)

Xin Wang, Zirui Chen, Haofen Wang, Leong Hou U, Zhao Li, Wenbin Guo

AI总结:

本文综述了大语言模型增强知识表示学习的方法,归纳了基于编码器、编码器-解码器和解码器三类技术路线,并梳理了下游任务应用与未来研究方向。

摘要:

知识表示学习(KRL)通过将知识事实投影到向量空间,对于使来自知识图谱(KG)的符号知识能够应用于下游任务至关重要。尽管KRL方法在建模知识图谱结构信息方面具有有效性,但它们正受到知识图谱稀疏性的困扰。基于Transformer架构的大语言模型(LLM)的兴起,为通过融入文本信息来解决知识图谱中的信息稀疏性问题、从而增强KRL提供了有前景的机会。LLM增强的KRL方法包括三种关键途径:利用详细上下文信息的基于编码器的方法、利用统一的Seq2Seq模型进行全面编码和解码的基于编码器-解码器的方法,以及利用来自大型语料库的广泛知识的基于解码器的方法。这些方法显著提升了KRL在应对广泛下游任务时的有效性和泛化能力。本工作提供了下游任务的广泛概述,同时识别了这些不断发展的领域中的新兴研究方向。

英文摘要:

Knowledge Representation Learning (KRL) is crucial for enabling applications of symbolic knowledge from Knowledge Graphs (KGs) to downstream tasks by projecting knowledge facts into vector spaces. Despite their effectiveness in modeling KG structural information, KRL methods are suffering from the sparseness of KGs. The rise of Large Language Models (LLMs) built on the Transformer architecture presents promising opportunities for enhancing KRL by incorporating textual information to address information sparsity in KGs. LLM-enhanced KRL methods, including three key approaches, encoder-based methods that leverage detailed contextual information, encoder-decoder-based methods that utilize a unified Seq2Seq model for comprehensive encoding and decoding, and decoder-based methods that utilize extensive knowledge from large corpora, have significantly advanced the effectiveness and generalization of KRL in addressing a wide range of downstream tasks. This work provides a broad overview of downstream tasks while simultaneously identifying emerging research directions in these evolving domains.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑