arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2401.06532cs.CLcs.IR
收藏

INTERS:通过指令微调释放大型语言模型在搜索中的潜力

INTERS: Unlocking the Power of Large Language Models in Search with Instruction Tuning

Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院);Beijing Academy of Artificial Intelligence(北京人工智能研究院);School of Computer Science, Beijing University of Posts and Telecommunications(北京邮电大学计算机学院);School of Artificial Intelligence, Nankai University(南开大学人工智能学院)

Yutao Zhu, Peitian Zhang, Chenghao Zhang, Yifei Chen, Binyu Xie, Zheng Liu, Ji-Rong Wen, Zhicheng Dou

AI总结:

本研究提出指令微调数据集INTERS,涵盖20个信息检索任务,显著提升LLaMA、Mistral、Phi等大型语言模型在查询理解、文档理解及关系理解上的性能。

摘要:

大型语言模型(LLMs)在各种自然语言处理任务中展现了令人印象深刻的能力。尽管如此,由于许多信息检索(IR)特定概念在自然语言中不常出现,将它们应用于信息检索任务仍然具有挑战性。虽然基于提示的方法可以为LLMs提供任务描述,但它们往往无法促进对IR任务的全面理解和执行,从而限制了LLMs的适用性。为了解决这一差距,在这项工作中,我们探索了指令微调在增强LLMs在IR任务中的熟练度方面的潜力。我们引入了一个新颖的指令微调数据集INTERS,涵盖了三个基本IR类别中的20个任务:查询理解、文档理解和查询-文档关系理解。数据来源于43个不同的数据集,并带有手动编写的模板。我们的实证结果表明,INTERS显著提升了各种公开可用的LLMs(如LLaMA、Mistral和Phi)在IR任务中的性能。此外,我们进行了广泛的实验,以分析指令设计、模板多样性、少样本演示和指令数量对性能的影响。我们在https://github.com/DaoD/INTERS公开提供我们的数据集和微调模型。

英文摘要:

Large language models (LLMs) have demonstrated impressive capabilities in various natural language processing tasks. Despite this, their application to information retrieval (IR) tasks is still challenging due to the infrequent occurrence of many IR-specific concepts in natural language. While prompt-based methods can provide task descriptions to LLMs, they often fall short in facilitating a comprehensive understanding and execution of IR tasks, thereby limiting LLMs' applicability. To address this gap, in this work, we explore the potential of instruction tuning to enhance LLMs' proficiency in IR tasks. We introduce a novel instruction tuning dataset, INTERS, encompassing 20 tasks across three fundamental IR categories: query understanding, document understanding, and query-document relationship understanding. The data are derived from 43 distinct datasets with manually written templates. Our empirical results reveal that INTERS significantly boosts the performance of various publicly available LLMs, such as LLaMA, Mistral, and Phi, in IR tasks. Furthermore, we conduct extensive experiments to analyze the effects of instruction design, template diversity, few-shot demonstrations, and the volume of instructions on performance. We make our dataset and the fine-tuned models publicly accessible at https://github.com/DaoD/INTERS.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑