arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.17897cs.CLcs.IR
收藏

一种用于本体中新概念放置的基于语言模型的框架

A Language Model based Framework for New Concept Placement in Ontologies

University of Oxford(牛津大学);University of Exeter(埃克塞特大学);University of Manchester(曼彻斯特大学);SNOMED International(SNOMED国际)

Hang Dong, Jiaoyan Chen, Yuan He, Yongsheng Gao, Ian Horrocks

AI总结:

本文提出三阶段语言模型框架,通过边搜索、形成丰富与选择完成本体新概念放置,并在 SNOMED CT/MedMentions 数据上验证微调 PLM 与 Cross-encoder 的效果。

摘要:

我们研究了利用语言模型将从文本中抽取的新概念插入本体的任务。我们探索了一种包含三个步骤的方法:边搜索,即寻找一组用于插入的候选位置(也就是概念之间的包含关系);边形成与丰富,利用本体结构生成并增强候选边;以及边选择,最终确定要放入的边。在所有步骤中,我们提出利用神经方法:对于边搜索,我们采用基于嵌入的方法以及结合 BERT 等预训练语言模型(PLM)的对比学习;对于边选择,我们调整了一种基于 BERT 微调的多标签 Edge-Cross-encoder,并使用 GPT 系列、FLAN-T5 和 Llama 2 等大型语言模型(LLM)。我们在近期使用 SNOMED CT 本体和 MedMentions 实体链接基准构建的数据集上评估了这些方法。本框架中的最佳设置使用微调后的 PLM 进行搜索,并使用多标签 Cross-encoder 进行选择。LLM 的零样本提示仍不足以胜任该任务,因此我们提出对 LLM 进行可解释的指令微调以提升性能。我们的研究展示了 PLM 的优势,并强调了 LLM 令人鼓舞的表现,这为未来研究提供了动力。

英文摘要:

We investigate the task of inserting new concepts extracted from texts into an ontology using language models. We explore an approach with three steps: edge search which is to find a set of candidate locations to insert (i.e., subsumptions between concepts), edge formation and enrichment which leverages the ontological structure to produce and enhance the edge candidates, and edge selection which eventually locates the edge to be placed into. In all steps, we propose to leverage neural methods, where we apply embedding-based methods and contrastive learning with Pre-trained Language Models (PLMs) such as BERT for edge search, and adapt a BERT fine-tuning-based multi-label Edge-Cross-encoder, and Large Language Models (LLMs) such as GPT series, FLAN-T5, and Llama 2, for edge selection. We evaluate the methods on recent datasets created using the SNOMED CT ontology and the MedMentions entity linking benchmark. The best settings in our framework use fine-tuned PLM for search and a multi-label Cross-encoder for selection. Zero-shot prompting of LLMs is still not adequate for the task, and we propose explainable instruction tuning of LLMs for improved performance. Our study shows the advantages of PLMs and highlights the encouraging performance of LLMs that motivates future studies.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑