通过激活引导提升语言模型的指令遵循能力
Improving Instruction-Following in Language Models through Activation Steering
AI总结：
本文提出通过计算含与不含指令输入的激活差异得到指令特定向量，以模块化激活引导方法增强语言模型对输出约束的遵守能力，实验验证其在有无显式指令场景及跨模型迁移中的有效性，为指令遵循提供推理时细粒度控制。
摘要：
语言模型的指令遵循能力对其众多实际应用至关重要。为深入探究并增强该能力，我们从语言模型中提取指令特定的向量表示，并用其引导模型。这些向量通过计算含指令与不含指令输入间的激活差异得到，实现了模块化的激活引导方法。我们证明该方法可增强模型对输出格式、长度、词汇包含等约束的遵守，提供指令遵循的推理时控制。在四个模型上的实验表明，即便无显式指令，我们也能利用激活向量引导模型遵循约束；当有指令时，该方法可提升性能。此外，我们探索了激活引导的组合性，成功同时应用多个指令。最后，我们证明在指令微调模型上计算的引导向量可迁移至提升基础模型性能。研究表明，激活引导为语言生成的细粒度控制提供了实用且可扩展的方法。代码与数据见https://github.com/microsoft/llm-steer-instruct。
英文摘要：
The ability to follow instructions is crucial for numerous real-world applications of language models. In pursuit of deeper insights and more powerful capabilities, we derive instruction-specific vector representations from language models and use them to steer models accordingly. These vectors are computed as the difference in activations between inputs with and without instructions, enabling a modular approach to activation steering. We demonstrate how this method can enhance model adherence to constraints such as output format, length, and word inclusion, providing inference-time control over instruction following. Our experiments across four models demonstrate how we can use the activation vectors to guide models to follow constraints even without explicit instructions and to enhance performance when instructions are present. Additionally, we explore the compositionality of activation steering, successfully applying multiple instructions simultaneously. Finally, we demonstrate that steering vectors computed on instruction-tuned models can transfer to improve base models. Our findings demonstrate that activation steering offers a practical and scalable approach for fine-grained control in language generation. Our code and data are available at https://github.com/microsoft/llm-steer-instruct.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
