arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.12847cs.CLcs.AIcs.LG
收藏

指令调优的语言模型是更好的知识学习者

Instruction-tuned Language Models are Better Knowledge Learners

Carnegie Mellon University(卡内基梅隆大学);FAIR at Meta(Meta FAIR研究院);University of Washington(华盛顿大学)

Zhengbao Jiang, Zhiqing Sun, Weijia Shi, Pedro Rodriguez, Chunting Zhou, Graham Neubig, Xi Victoria Lin, Wen-tau Yih, Srinivasan Iyer

AI总结:

针对LLM持续学习新知识时标准流程(先文档预训练后指令调优)效果不佳的问题,提出预指令调优(PIT)方法,先问答后文档训练,显著提升知识吸收能力,性能提升17.8%。

摘要:

为了使基于大型语言模型(LLM)的助手能够有效适应不断变化的信息需求,必须能够通过持续对新数据进行训练来更新其事实知识。标准做法是先对新文档进行持续预训练,然后在问答(QA)对上进行指令调优。然而,我们发现,尽管文档的困惑度被最小化,但采用这种配方训练的LLM在回答问题时却表现挣扎。我们发现,QA对通常较为直接,而文档则更为复杂,将许多事实陈述错综复杂地编织在一起。因此,我们假设,在文档持续预训练之前先让LLM接触QA对是有益的,这样从复杂文档中编码知识的过程就会考虑到如何通过问题访问这些知识。基于此,我们提出了预指令调优(PIT)方法,即在训练文档之前先对问题进行指令调优。这与标准的指令调优形成对比,后者是在文档训练之后学习如何提取知识。大量实验和消融研究表明,预指令调优显著增强了LLM从新文档中吸收知识的能力,比标准指令调优高出17.8%。

英文摘要:

In order for large language model (LLM)-based assistants to effectively adapt to evolving information needs, it must be possible to update their factual knowledge through continued training on new data. The standard recipe for doing so involves continued pre-training on new documents followed by instruction-tuning on question-answer (QA) pairs. However, we find that LLMs trained with this recipe struggle to answer questions, even though the perplexity of documents is minimized. We found that QA pairs are generally straightforward, while documents are more complex, weaving many factual statements together in an intricate manner. Therefore, we hypothesize that it is beneficial to expose LLMs to QA pairs before continued pre-training on documents so that the process of encoding knowledge from complex documents takes into account how this knowledge is accessed through questions. Based on this, we propose pre-instruction-tuning (PIT), a method that instruction-tunes on questions prior to training on documents. This contrasts with standard instruction-tuning, which learns how to extract knowledge after training on documents. Extensive experiments and ablation studies demonstrate that pre-instruction-tuning significantly enhances the ability of LLMs to absorb knowledge from new documents, outperforming standard instruction-tuning by 17.8%.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑