通过反向传播编程：在微调LLMs时，一条指令的价值相当于100个示例
Programming by Backprop: An Instruction is Worth 100 Examples When Finetuning LLMs
AI总结：
通过反向传播编程（PBB）使LLMs能从声明性指令中学习程序性知识，提升训练效率并减少对大量示例的依赖。
摘要：
大型语言模型（LLMs）通常通过演示或经验来学习行为，但其大部分训练数据都是声明性的：指令、规则和描述，这些指定了行为但不展示如何执行。我们引入了通过反向传播编程（PBB）：一种训练制度，使LLMs能够从训练过程中遇到的声明性指令中获取程序性知识（即可重用的行为）。通过PBB，训练数据中的指令为模型权重提供了“编程”特定行为的机会。PBB的核心原理是将学习指令如何映射到行为与内化新指令分开。我们设计了两种不同的PBB课程，利用这一原理。通过在两个领域（从Python源代码执行算法和从上下文无关文法生成文本）中的受控实验，我们展示了这些课程相对于训练于同质数据混合物的益处。关键的是，PBB高度样本高效，单条指令可替代多达100个执行示例。尽管训练数据中的指令执行仍然不如在上下文中的指令可靠，但我们的结果表明，通过PBB可以将程序性知识以嘈杂的方式“编程”进LLMs，这对数据管理和安全性具有重要意义。
英文摘要：
Large language models (LLMs) are typically trained to acquire behaviours from demonstrations or experience, yet much of their training data is declarative: instructions, rules, and descriptions that specify behaviours without showing how to execute them. We introduce Programming by Backprop (PBB): a training regime that enables LLMs to acquire procedural knowledge (i.e., reusable behaviours) from declarative instructions encountered during training. With PBB, instructions in training data provide an opportunity to `program' specific behaviours into model weights. The core principle underpinning PBB is the separation of learning how instructions map to behaviour from internalising new instructions. We devise two distinct PBB curricula that leverage this principle. Through controlled experiments across two domains (algorithmic execution from Python source code and text generation from context-free grammars), we demonstrate the benefit of these curricula over training on a homogeneous data mixture. Crucially, PBB is highly sample efficient, with a single instruction substituting for up to 100 execution examples. Though execution of instructions in training data remains less reliable than when instructions are given in-context, our results demonstrate that procedural knowledge can be noisily `programmed' into LLMs through PBB, with important implications for data curation and safety.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
