上下文学习(In-Context Learning, ICL)和指令微调(Instruction Tuning, IT)是将大型语言模型(Large Language Models, LLMs)应用于下游任务的两大主要范式。然而,它们之间存在显著差异。在ICL中,推理时会提供一组演示(demonstrations),但LLM的参数不会更新。在IT中,训练时会使用一组演示来调整LLM的参数,但推理时不使用任何演示。尽管越来越多的文献探讨了ICL和IT,但关于这些主题的研究在很大程度上是孤立进行的,导致这两种范式之间缺乏联系。在本工作中,我们通过考察LLM在这两种范式中隐藏状态的变化来探索ICL与IT之间的关系。通过使用LLaMA-2(7B和13B)进行的精心设计的实验,我们发现ICL是隐式的IT。换言之,ICL改变LLM的隐藏状态,其方式如同这些演示被用于对模型进行指令微调一样。此外,ICL与IT之间的趋同在很大程度上取决于与所提供的演示相关的若干因素。总体而言,本工作为探索ICL与IT之间的联系提供了独特视角,并有助于理解LLM的行为。
英文摘要:
In-Context Learning (ICL) and Instruction Tuning (IT) are two primary paradigms of adopting Large Language Models (LLMs) to downstream applications. However, they are significantly different. In ICL, a set of demonstrations are provided at inference time but the LLM's parameters are not updated. In IT, a set of demonstrations are used to tune LLM's parameters in training time but no demonstrations are used at inference time. Although a growing body of literature has explored ICL and IT, studies on these topics have largely been conducted in isolation, leading to a disconnect between these two paradigms. In this work, we explore the relationship between ICL and IT by examining how the hidden states of LLMs change in these two paradigms. Through carefully designed experiments conducted with LLaMA-2 (7B and 13B), we find that ICL is implicit IT. In other words, ICL changes an LLM's hidden states as if the demonstrations were used to instructionally tune the model. Furthermore, the convergence between ICL and IT is largely contingent upon several factors related to the provided demonstrations. Overall, this work offers a unique perspective to explore the connection between ICL and IT and sheds light on understanding the behaviors of LLM.
Building Instruction-Tuning Datasets from Human-Written Instructions with Open-Weight Large Language Models
Department of Computer Science, School of Computing, Institute of Science Tokyo(计算机科学系、计算学院、科学东京研究所);National Institute of Advanced Industrial Science and Technology(国家先进工业科学与技术研究院);Research and Development Center for Large Language Models, NII(大型语言模型研究与开发中心、日本科学振兴会)
A Survey on Data Selection for LLM Instruction Tuning
Harbin Institute of Technology(哈尔滨工业大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Harbin Institute of Technology (Weihai)(哈尔滨工业大学(威海))
MLAN: Language-Based Instruction Tuning Preserves and Transfers Knowledge in Multimodal Language Models
Washington University in St. Louis(华盛顿大学圣路易斯分校);The University of British Columbia(不列颠哥伦比亚大学);Google Research(谷歌研究);Virginia Tech(弗吉尼亚理工大学);University of California, Davis(加州大学戴维斯分校);Sony AI(索尼人工智能);University of California, Berkeley(加州大学伯克利分校)
Synthesizing Instruction-Tuning Datasets with Contrastive Decoding
Department of Computer Science, School of Computing, Institute of Science Tokyo(东京科学大学计算机科学系);National Institute of Advanced Industrial Science and Technology(国家先进工业科学与技术研究院);Research and Development Center for Large Language Models, NII(大型语言模型研究与开发中心,日本信息产业技术综合研究所)
CLASS-IT: Conversational and Lecture-Aligned Small-Scale Instruction Tuning for BabyLMs
CoLing Lab, Department of Philology, Literature and Linguistics, University of Pisa(皮尔切大学哲学、文学与语言学系语言实验室);Department of Computer Science, University of Pisa(皮尔切大学计算机科学系)
Specialist or Generalist? Instruction Tuning for Specific NLP Tasks
Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);Language Technology Lab, University of Cambridge(剑桥大学语言技术实验室);Cohere;Tencent AI Lab(腾讯AI实验室)
PACIT: Unlocking the Power of Examples for Better In-Context Instruction Tuning
Nanjing University(南京大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Southern University of Science and Technology(南方科技大学);Shanghai University of Finance and Economics(上海财经大学)
Mixture-of-Experts Meets Instruction Tuning:A Winning Combination for Large Language Models
Google(谷歌);University of California, Berkeley(加州大学伯克利分校);Massachusetts Institute of Technology(麻省理工学院);University of Massachusetts Amherst(马萨诸塞大学阿默斯特分校);The University of Texas at Austin(得克萨斯大学奥斯汀分校)
Multilingual Pretraining and Instruction Tuning Improve Cross-Lingual Knowledge Alignment, But Only Shallowly
National Key Laboratory for Novel Software Technology, Nanjing University(南京大学计算机软件新技术国家重点实验室);Department of Linguistics and Translation, City University of Hong Kong(香港城市大学语言学及翻译学系)
Minimal-Edit Instruction Tuning for Low-Resource Indic GEC
Indian Heritage Language Computing Team Special and Strategic Projects (SSP) Group(印度遗产语言计算团队特殊与战略项目组);Centre for Development of Advanced Computing (C DAC)(高级计算发展中心)