arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2304.14856cs.IR
收藏

基于提示学习的知识密集型语言任务统一生成式检索器

A Unified Generative Retriever for Knowledge-Intensive Language Tasks via Prompt Learning

Jiangui Chen, Ruqing Zhang, Jiafeng Guo, Maarten de Rijke, Yiqun Liu, Yixing Fan, Xueqi Cheng

AI总结:

针对知识密集型语言任务专用检索器泛化差、部署成本高的问题,提出统一生成式检索器UGR,通过基于n-gram的标识符统一检索形式、提示学习适配不同任务,在KILT基准上验证了其有效性与泛化能力。

摘要:

知识密集型语言任务(KILTs)可从大型外部知识语料库中检索高质量相关上下文而受益。学习能返回合适语义粒度相关上下文的任务专用检索器,例如文档检索器、段落检索器、句子检索器和实体检索器,或有助于提升端到端任务的性能。但任务专用检索器通常对新领域和新任务的泛化能力较差,且在实际中部署多种专用检索器的成本可能较高。我们提出了一种统一生成式检索器(UGR),它兼具任务专用的有效性与在KILTs不同检索任务上的稳健性能。为实现这一目标,我们做出两项主要贡献:(i)为将不同检索任务统一为单一生成形式,我们引入了一种基于n-gram的标识符,用于标识KILTs中不同粒度级别的相关上下文。(ii)为用单个模型处理不同检索任务,我们采用提示学习策略,并研究了为每个任务设计提示token的三种方法。通过这种方式,所提出的UGR模型不仅能在不同任务间共享通用知识以实现更好的泛化,还能通过区分任务特有特征有效执行不同的检索任务。我们在一组异构检索语料库上,以监督和多任务的方式,结合精心设计的提示对UGR进行训练。在KILT基准上的实验结果证明了UGR在域内数据集、域外数据集以及未见过的任务上均具有有效性。

英文摘要:

Knowledge-intensive language tasks (KILTs) benefit from retrieving high-quality relevant contexts from large external knowledge corpora. Learning task-specific retrievers that return relevant contexts at an appropriate level of semantic granularity, such as a document retriever, passage retriever, sentence retriever, and entity retriever, may help to achieve better performance on the end-to-end task. But a task-specific retriever usually has poor generalization ability to new domains and tasks, and it may be costly to deploy a variety of specialised retrievers in practice. We propose a unified generative retriever (UGR) that combines task-specific effectiveness with robust performance over different retrieval tasks in KILTs. To achieve this goal, we make two major contributions: (i) To unify different retrieval tasks into a single generative form, we introduce an n-gram-based identifier for relevant contexts at different levels of granularity in KILTs. And (ii) to address different retrieval tasks with a single model, we employ a prompt learning strategy and investigate three methods to design prompt tokens for each task. In this way, the proposed UGR model can not only share common knowledge across tasks for better generalization, but also perform different retrieval tasks effectively by distinguishing task-specific characteristics. We train UGR on a heterogeneous set of retrieval corpora with well-designed prompts in a supervised and multi-task fashion. Experimental results on the KILT benchmark demonstrate the effectiveness of UGR on in-domain datasets, out-of-domain datasets, and unseen tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑