arXiv 2305.111592023-05-18cs.CL

对齐指令任务解锁大型语言模型作为零样本关系抽取器

Aligning Instruction Tasks Unlocks Large Language Models as Zero-Shot Relation Extractors

The Ohio State University（俄亥俄州立大学）

Kai Zhang, Bernal Jiménez Gutiérrez, Yu Su

备注：: ACL 2023 Findings; The code is available at https://github.com/OSU-NLP-Group/QA4RE

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对大型语言模型在关系抽取任务上表现不佳的问题，提出QA4RE框架，通过将关系抽取与问答任务对齐，显著提升了模型在零样本和少样本设置下的性能与鲁棒性。

摘要：

近期研究表明，在大型指令遵循数据集上微调大型语言模型（LLMs）能显著提升其在广泛NLP任务上的表现，尤其是在零样本设置下。然而，即使是先进的指令微调LLM，在关系抽取（RE）这一基础信息抽取任务上，仍无法超越小型语言模型（LM）。我们假设，由于RE在指令微调数据集中占比极低（Wang等人，2022年指出其不足所有任务的1%），指令微调未能激发LLM的强大RE能力。为解决此局限，我们提出QA4RE框架，将RE与指令微调数据集中的主导任务——问答（QA）进行对齐。在四个数据集上使用两个系列的指令微调LLM（共六个LLM）进行的全面零样本RE实验表明，我们的QA4RE框架持续提升了LLM性能，强有力地验证了我们的假设，并使LLM以大幅优势超越了强大的零样本基线。此外，我们提供了详尽的实验与讨论，以展示QA4RE框架的鲁棒性、少样本有效性和强大的可迁移性。本研究展示了一种极具前景的方法，即将LLM适配至具有挑战性且代表性不足的任务，通过将这些任务与QA等更常见的指令微调任务进行对齐来实现。

英文摘要：

Recent work has shown that fine-tuning large language models (LLMs) on large-scale instruction-following datasets substantially improves their performance on a wide range of NLP tasks, especially in the zero-shot setting. However, even advanced instruction-tuned LLMs still fail to outperform small LMs on relation extraction (RE), a fundamental information extraction task. We hypothesize that instruction-tuning has been unable to elicit strong RE capabilities in LLMs due to RE's low incidence in instruction-tuning datasets, making up less than 1% of all tasks (Wang et al., 2022). To address this limitation, we propose QA4RE, a framework that aligns RE with question answering (QA), a predominant task in instruction-tuning datasets. Comprehensive zero-shot RE experiments over four datasets with two series of instruction-tuned LLMs (six LLMs in total) demonstrate that our QA4RE framework consistently improves LLM performance, strongly verifying our hypothesis and enabling LLMs to outperform strong zero-shot baselines by a large margin. Additionally, we provide thorough experiments and discussions to show the robustness, few-shot effectiveness, and strong transferability of our QA4RE framework. This work illustrates a promising way of adapting LLMs to challenging and underrepresented tasks by aligning these tasks with more common instruction-tuning tasks like QA.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。