arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2307.00342cs.CLcs.IR
收藏

通过促进任务特化改进多任务检索

Improving Multitask Retrieval by Promoting Task Specialization

Rutgers University(罗格斯大学);Microsoft(微软)

Wenzheng Zhang, Chenyan Xiong, Karl Stratos, Arnold Overwijk

AI总结:

本文提出通过选择多任务优化的预训练模型与自适应学习方法促进任务特化,训练出在 KILT 基准上优于任务特定检索器的多任务检索器。

摘要:

在多任务检索中,单个检索器被训练用于为多个任务检索相关上下文。尽管具有实际吸引力,但朴素的多任务检索落后于任务特定检索(即为每个任务训练单独的检索器)。我们表明,通过促进任务特化,可以训练出优于任务特定检索器的多任务检索器。主要要素包括:(1)更好地选择预训练模型(明确为多任务优化的模型)以及兼容的提示;(2)一种新颖的自适应学习方法,鼓励每个参数特化于特定任务。由此产生的多任务检索器在 KILT 基准上表现优异。经分析,我们发现与没有提示或自适应学习的朴素多任务处理相比,该模型确实学习了更具任务特化性的参数。

英文摘要:

In multitask retrieval, a single retriever is trained to retrieve relevant contexts for multiple tasks. Despite its practical appeal, naive multitask retrieval lags behind task-specific retrieval in which a separate retriever is trained for each task. We show that it is possible to train a multitask retriever that outperforms task-specific retrievers by promoting task specialization. The main ingredients are: (1) a better choice of pretrained model (one that is explicitly optimized for multitasking) along with compatible prompting, and (2) a novel adaptive learning method that encourages each parameter to specialize in a particular task. The resulting multitask retriever is highly performant on the KILT benchmark. Upon analysis, we find that the model indeed learns parameters that are more task-specialized compared to naive multitasking without prompting or adaptive learning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑