arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 2207.03030cs.CLcs.IR
收藏

基于相关抽样的多任务检索增强文本生成

Multi-Task Retrieval-Augmented Text Generation with Relevance Sampling

TU Wien(维也纳工业大学);Google(谷歌);University of Massachusetts Amherst(马萨诸塞大学阿默斯特分校)

Sebastian Hofstätter, Jiecao Chen, Karthik Raman, Hamed Zamani

摘要:

研究了面向知识密集型任务的检索增强生成模型的多任务训练问题。我们建议通过利用知识密集型生成的一个独特属性来清理训练集:查询-答案对与知识库中的项目的连接。我们过滤训练样本通过相关性标签的置信度阈值,无论一对是由知识库回答与否。我们训练一个单一的融合解码器(FiD)发生器的七个组合任务的KILT基准。实验结果表明,我们简单而有效的方法大大改善了两个严重不平衡任务的竞争基线;并且在剩余任务上显示出较小的改善或没有显着的回归。此外,我们证明了我们的多任务训练与相关标签采样尺度以及增加的模型容量,并实现了最先进的结果,在五个七个KILT任务。

英文摘要:

This paper studies multi-task training of retrieval-augmented generation models for knowledge-intensive tasks. We propose to clean the training set by utilizing a distinct property of knowledge-intensive generation: The connection of query-answer pairs to items in the knowledge base. We filter training examples via a threshold of confidence on the relevance labels, whether a pair is answerable by the knowledge base or not. We train a single Fusion-in-Decoder (FiD) generator on seven combined tasks of the KILT benchmark. The experimental results suggest that our simple yet effective approach substantially improves competitive baselines on two strongly imbalanced tasks; and shows either smaller improvements or no significant regression on the remaining tasks. Furthermore, we demonstrate our multi-task training with relevance label sampling scales well with increased model capacity and achieves state-of-the-art results in five out of seven KILT tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑