arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2404.17809cs.CLcs.AI
收藏

召回、检索与推理:迈向更好的上下文关系抽取

Recall, Retrieve and Reason: Towards Better In-Context Relation Extraction

School of Computer Science and Engineering, Southeast University(东南大学计算机科学与工程学院);Beijing Institute of Computer Technology and Application(北京计算机技术及应用研究所);Interdisciplinary Applications (Southeast University), Ministry of Education(教育部东南大学交叉应用中心)

Guozheng Li, Peng Wang, Wenjun Ke, Yikai Guo, Ke Ji, Ziyu Shang, Jiajun Liu, Zijie Xu

AI总结:

本文提出一种召回-检索-推理框架,通过提取本体知识生成有效实体对查询并检索训练样本作为示范,结合指令微调提升LLMs的上下文学习能力,在句子级关系抽取任务上取得了有竞争力或新的最先进性能。

摘要:

关系抽取(RE)旨在识别文本中提及的实体之间的关系。尽管大型语言模型(LLMs)在各种任务中展示了令人印象深刻的上下文学习(ICL)能力,但与大多数有监督微调的RE方法相比,它们的性能仍然较差。在LLMs中利用ICL进行RE面临两个挑战:(1)从训练样本中检索好的示范,以及(2)使LLMs在RE中表现出强大的ICL能力。一方面,在RE中检索好的示范是一个复杂的过程,容易导致实体和关系的相关性较低。另一方面,使用LLM进行ICL在RE中表现不佳,因为RE本质上不同于语言建模,或者LLM不够大。在这项工作中,我们提出了一种新颖的召回-检索-推理RE框架,该框架将LLMs与检索语料库(训练样本)协同起来,以实现相关检索和可靠的上下文推理。具体来说,我们从训练数据集中提取一致的本体知识,让LLMs生成基于检索语料库的相关实体对作为有效查询。然后使用这些实体对从检索语料库中检索相关的训练样本,作为示范,通过指令微调使LLMs进行更好的ICL。在不同LLMs和RE数据集上的大量实验表明,我们的方法生成相关且有效的实体对,并提升了LLMs的ICL能力,在句子级RE上与先前的有监督微调方法和基于ICL的方法相比,取得了有竞争力或新的最先进性能。

英文摘要:

Relation extraction (RE) aims to identify relations between entities mentioned in texts. Although large language models (LLMs) have demonstrated impressive in-context learning (ICL) abilities in various tasks, they still suffer from poor performances compared to most supervised fine-tuned RE methods. Utilizing ICL for RE with LLMs encounters two challenges: (1) retrieving good demonstrations from training examples, and (2) enabling LLMs exhibit strong ICL abilities in RE. On the one hand, retrieving good demonstrations is a non-trivial process in RE, which easily results in low relevance regarding entities and relations. On the other hand, ICL with an LLM achieves poor performance in RE while RE is different from language modeling in nature or the LLM is not large enough. In this work, we propose a novel recall-retrieve-reason RE framework that synergizes LLMs with retrieval corpora (training examples) to enable relevant retrieving and reliable in-context reasoning. Specifically, we distill the consistently ontological knowledge from training datasets to let LLMs generate relevant entity pairs grounded by retrieval corpora as valid queries. These entity pairs are then used to retrieve relevant training examples from the retrieval corpora as demonstrations for LLMs to conduct better ICL via instruction tuning. Extensive experiments on different LLMs and RE datasets demonstrate that our method generates relevant and valid entity pairs and boosts ICL abilities of LLMs, achieving competitive or new state-of-the-art performance on sentence-level RE compared to previous supervised fine-tuning methods and ICL-based methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑