KV缓存引导：控制冻结大语言模型
KV Cache Steering for Controlling Frozen LLMs
AI总结：
我们提出缓存引导，一种对键值缓存施加一次性干预的轻量方法，可诱导小模型进行思维链推理，无需微调或改提示，并在推理基准上提升性能，支持推理风格可控迁移。
摘要：
我们提出缓存引导（cache steering），一种轻量级方法，通过对键值缓存直接施加一次性干预来实现语言模型的隐式引导。为验证其有效性，我们将缓存引导应用于诱导小语言模型进行思维链推理。我们的方法从推理轨迹中构建引导向量，这些轨迹可来自教师模型（如GPT-4o）或现有的人类标注，从而在不进行微调或修改提示的情况下，将模型行为转向更明确、多步骤的推理。在多种推理基准上的实验评估表明，缓存引导改善了模型推理的定性结构和定量任务性能。额外实验显示，该方法也能扩展到更大的模型，并在GPQA和MATH等具有挑战性的数据集上取得进一步提升。与需要持续干预的先前激活引导技术相比，我们的一次性缓存引导在推理延迟、超参数稳定性和与现有推理API集成的便捷性方面具有显著优势。除了单纯的推理诱导，我们还展示了缓存引导能够实现推理风格（如逐步、因果、类比）的可控迁移，使其成为语言模型行为层面引导的实用工具。
英文摘要：
We propose cache steering, a lightweight method for implicit steering of language models via a one-shot intervention applied directly to the key-value cache. To validate its effectiveness, we apply cache steering to induce chain-of-thought reasoning in small language models. Our approach constructs steering vectors from reasoning traces, obtained either from teacher models (e.g., GPT-4o) or existing human annotations, that shift model behavior toward more explicit, multi-step reasoning without fine-tuning or prompt modifications. Experimental evaluations on diverse reasoning benchmarks demonstrate that cache steering improves both the qualitative structure of model reasoning and quantitative task performance. Additional experiments show that the method also scales to larger models and yields further gains on challenging datasets such as GPQA and MATH. Compared to prior activation steering techniques that require continuous interventions, our one-shot cache steering offers substantial advantages in terms of inference latency, hyperparameter stability, and ease of integration with existing inference APIs. Beyond mere reasoning induction, we show that cache steering enables controllable transfer of reasoning styles (e.g., stepwise, causal, analogical), making it a practical tool for behavior-level guidance of language models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
