无痛激活导向：一种自动化、轻量级的微调大型语言模型方法
Painless Activation Steering: An Automated, Lightweight Approach for Post-Training Large Language Models
AI总结：
本文提出Painless Activation Steering，一种自动化方法，无需人工干预即可利用标注数据提升模型性能，尤其在行为任务中表现优异，但对智能任务效果有限。
摘要：
语言模型通常通过权重或提示导向进行微调，但前者耗时昂贵，后者控制不精确且需手动试错。激活导向（AS）提供了一种更经济、快速且可控的替代方法，但现有技术需人工构造提示对或进行大量特征标注，不如RL和SFT等方法方便。本文引入Painless Activation Steering（PAS），一种完全自动的方法，可利用任何标注数据集进行AS，无需提示构造、特征标注或人工干预。在三个开源模型和18个任务上评估PAS，发现其在行为任务中性能可靠，但对智能任务效果有限。 introspective variant（iPAS）在偏差、道德和对齐任务上分别提升了10.1%、5.2%和34.8%。此外，PAS在上下文学习（ICL）和SFT基础上还提供了额外增益。PAS构建了一个快速、轻量的激活向量，可低成本训练、存储和激活。实验结果为AS的应用提供了明确的指导，展示了其作为实用自动化微调方法的潜力。
英文摘要：
Language models (LMs) are typically post-trained for desired capabilities and behaviors via weight-based or prompt-based steering, but the former is time-consuming and expensive, and the latter is not precisely controllable and often requires manual trial-and-error. While activation steering (AS) promises a cheap, fast, and controllable alternative to the two existing post-training methods, current AS techniques require hand-crafted prompt pairs or labor-intensive feature annotation, making them more inconvenient than the plug-and-play methods such as Reinforcement Learning (RL) and Supervised Fine-Tuning (SFT). We introduce Painless Activation Steering (PAS), a family of fully automated methods that make AS readily usable with any given labeled dataset, with no need for prompt construction, feature labeling, or human intervention. We evaluate PAS on three open-weight models (Llama3.1-8B-Instruct, DeepSeek-R1-Distill-8B, and Nous-Hermes-2) and 18 tasks; we find that PAS reliably improves performance for behavior tasks, but not for intelligence-oriented tasks. The introspective variant (iPAS) delivers the strongest causal steering effects (10.1% on Bias, 5.2% on Morality, and 34.8% on Alignment). We also show PAS delivers additional gains on top of In-Context Learning (ICL) and SFT. PAS constructs a fast, lightweight activation vector that can be cheaply trained, easily stored, and activated at will. Our results provide a characterization of where AS helps, where it fails, and how to deploy it as a practical, automated LM post-training option.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
