arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2311.10367cs.CL
收藏

探索上下文学习与指令微调之间的关系

Exploring the Relationship between In-Context Learning and Instruction Tuning

University of Notre Dame(圣母大学)

Hanyu Duan, Yixuan Tang, Yi Yang, Ahmed Abbasi, Kar Yan Tam

AI总结:

本文通过分析LLaMA-2模型隐藏状态的变化,发现上下文学习(ICL)本质上是隐式的指令微调(IT),且两者趋同程度取决于演示因素,为理解LLM行为提供了新视角。

摘要:

上下文学习(In-Context Learning, ICL)和指令微调(Instruction Tuning, IT)是将大型语言模型(Large Language Models, LLMs)应用于下游任务的两大主要范式。然而,它们之间存在显著差异。在ICL中,推理时会提供一组演示(demonstrations),但LLM的参数不会更新。在IT中,训练时会使用一组演示来调整LLM的参数,但推理时不使用任何演示。尽管越来越多的文献探讨了ICL和IT,但关于这些主题的研究在很大程度上是孤立进行的,导致这两种范式之间缺乏联系。在本工作中,我们通过考察LLM在这两种范式中隐藏状态的变化来探索ICL与IT之间的关系。通过使用LLaMA-2(7B和13B)进行的精心设计的实验,我们发现ICL是隐式的IT。换言之,ICL改变LLM的隐藏状态,其方式如同这些演示被用于对模型进行指令微调一样。此外,ICL与IT之间的趋同在很大程度上取决于与所提供的演示相关的若干因素。总体而言,本工作为探索ICL与IT之间的联系提供了独特视角,并有助于理解LLM的行为。

英文摘要:

In-Context Learning (ICL) and Instruction Tuning (IT) are two primary paradigms of adopting Large Language Models (LLMs) to downstream applications. However, they are significantly different. In ICL, a set of demonstrations are provided at inference time but the LLM's parameters are not updated. In IT, a set of demonstrations are used to tune LLM's parameters in training time but no demonstrations are used at inference time. Although a growing body of literature has explored ICL and IT, studies on these topics have largely been conducted in isolation, leading to a disconnect between these two paradigms. In this work, we explore the relationship between ICL and IT by examining how the hidden states of LLMs change in these two paradigms. Through carefully designed experiments conducted with LLaMA-2 (7B and 13B), we find that ICL is implicit IT. In other words, ICL changes an LLM's hidden states as if the demonstrations were used to instructionally tune the model. Furthermore, the convergence between ICL and IT is largely contingent upon several factors related to the provided demonstrations. Overall, this work offers a unique perspective to explore the connection between ICL and IT and sheds light on understanding the behaviors of LLM.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑