arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2301.02458cs.CLcs.LG
收藏

主题作为实体簇:基于大型语言模型和图神经网络的实体主题

Topics as Entity Clusters: Entity-based Topics from Large Language Models and Graph Neural Networks

Huawei Ireland Research Centre(华为爱尔兰研究中心)

Manuel V. Loureiro, Steven Derby, Tri Kurniawan Wijaya

AI总结:

本文探索基于实体的神经主题建模,提出一种利用大型语言模型和图神经网络提取实体双峰向量表示的新型主题聚类方法,实验证明其在处理实体时优于现有模型。

摘要:

主题模型旨在揭示文本语料库中的潜在结构,通常通过对文档的词袋表示使用词频统计来实现。近年来,概念实体——可解释的、与外部知识资源相关的语言无关特征——已被用来替代词级标记,因为单词通常需要大量的语言处理,且对可解释性的保证极少。然而,当前文献在探索纯实体驱动的神经主题建模方面存在局限。例如,尽管使用实体来引出主题结构具有优势,但目前尚不清楚现有技术是否兼容这些组织稀疏、信息密集的概念单元。在这项工作中,我们探索了基于实体的神经主题建模,并提出了一种使用实体双峰向量表示的新型主题聚类方法。具体而言,我们从大型语言模型和在符号关系知识库上训练的图神经网络中提取这些潜在表示,以推导这些概念单元最显著的方面。对连贯性指标的分析证实,与最先进的模型相比,我们的方法更适合处理实体,特别是当使用在知识库上训练的基于图的嵌入时。

英文摘要:

Topic models aim to reveal latent structures within a corpus of text, typically through the use of term-frequency statistics over bag-of-words representations from documents. In recent years, conceptual entities -- interpretable, language-independent features linked to external knowledge resources -- have been used in place of word-level tokens, as words typically require extensive language processing with a minimal assurance of interpretability. However, current literature is limited when it comes to exploring purely entity-driven neural topic modeling. For instance, despite the advantages of using entities for eliciting thematic structure, it is unclear whether current techniques are compatible with these sparsely organised, information-dense conceptual units. In this work, we explore entity-based neural topic modeling and propose a novel topic clustering approach using bimodal vector representations of entities. Concretely, we extract these latent representations from large language models and graph neural networks trained on a knowledge base of symbolic relations, in order to derive the most salient aspects of these conceptual units. Analysis of coherency metrics confirms that our approach is better suited to working with entities in comparison to state-of-the-art models, particularly when using graph-based embeddings trained on a knowledge base.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑