模型能从示例中学习技能组合吗？
Can Models Learn Skill Composition from Examples?
AI总结：
本文借鉴SKILL-MIX设置，通过微调7B和13B模型于GPT-4生成的技能组合文本，发现模型能从k=2、3示例泛化至k=4、5组合，并提升对未见技能的组合能力，表明技能丰富文本可增强组合泛化。
摘要：
随着大型语言模型（LLM）变得越来越先进，其展现组合泛化能力——即在训练中未遇到的新颖方式下组合所学技能的能力——已引起广泛关注。这种泛化类型，尤其是在超出训练数据的场景中，在人工智能安全与对齐研究中也具有重大意义。一项近期研究引入了SKILL-MIX评估，其中模型被要求撰写一段短文，展示指定$k$元组语言技能的使用。虽然小模型即使在$k=3$时也难以完成组合，但像GPT-4这样的大型模型在$k=5$和$6$时表现相当不错。\n\n在本文中，我们采用类似于SKILL-MIX的设置来评估较小模型从示例中学习组合泛化的能力。利用多样化的语言技能集——包括修辞、文学、推理、心智理论和常识——我们使用GPT-4生成展示随机$k$技能子集的文本样本。随后在7B和13B参数模型上对这些组合技能文本进行微调，随着$k$值增加，揭示了以下发现：（1）在$k=2$和$3$技能组合上训练，显著提高了模型以$k=4$和$5$技能撰写文本的能力，尽管模型在训练中从未见过此类示例。（2）当技能类别被分为训练组和留出组时，模型在测试时使用留出技能撰写文本的能力显著提升，尽管微调期间仅见过训练技能，这证明了该训练方法即使对先前未见技能也有效。这项研究还表明，将富含技能（可能是合成的）文本纳入训练可以显著增强模型的组合能力。
英文摘要：
As large language models (LLMs) become increasingly advanced, their ability to exhibit compositional generalization -- the capacity to combine learned skills in novel ways not encountered during training -- has garnered significant attention. This type of generalization, particularly in scenarios beyond training data, is also of great interest in the study of AI safety and alignment. A recent study introduced the SKILL-MIX evaluation, where models are tasked with composing a short paragraph demonstrating the use of a specified $k$-tuple of language skills. While small models struggled with composing even with $k=3$, larger models like GPT-4 performed reasonably well with $k=5$ and $6$. In this paper, we employ a setup akin to SKILL-MIX to evaluate the capacity of smaller models to learn compositional generalization from examples. Utilizing a diverse set of language skills -- including rhetorical, literary, reasoning, theory of mind, and common sense -- GPT-4 was used to generate text samples that exhibit random subsets of $k$ skills. Subsequent fine-tuning of 7B and 13B parameter models on these combined skill texts, for increasing values of $k$, revealed the following findings: (1) Training on combinations of $k=2$ and $3$ skills results in noticeable improvements in the ability to compose texts with $k=4$ and $5$ skills, despite models never having seen such examples during training. (2) When skill categories are split into training and held-out groups, models significantly improve at composing texts with held-out skills during testing despite having only seen training skills during fine-tuning, illustrating the efficacy of the training approach even with previously unseen skills. This study also suggests that incorporating skill-rich (potentially synthetic) text into training can substantially enhance the compositional capabilities of models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
