通过目标干预对语言模型进行多属性引导
Multi-Attribute Steering of Language Models via Targeted Intervention
AI总结：
研究如何对语言模型进行多属性引导，提出MAT-Steer框架，通过对齐目标学习引导向量，减少属性间冲突，在问答和生成任务中评估，该框架优于现有方法。
摘要：
推理时干预（ITI）已成为一种有前景的方法，可通过干预令牌表示来引导大语言模型（LLM）行为朝特定方向发展，且无需对LLM参数进行昂贵更新。但现有ITI方法无法扩展到存在冲突的多属性设置。为此引入多属性目标引导（MAT-Steer）框架，通过对齐目标学习引导向量，减少属性间冲突。在问答和生成任务中评估，MAT-Steer优于现有方法。
英文摘要：
Inference-time intervention (ITI) has emerged as a promising method for steering large language model (LLM) behavior in a particular direction (e.g., improving helpfulness) by intervening on token representations without costly updates to the LLM's parameters. However, existing ITI approaches fail to scale to multi-attribute settings with conflicts, such as enhancing helpfulness while also reducing toxicity. To address this, we introduce Multi-Attribute Targeted Steering (MAT-Steer), a novel steering framework designed for selective token-level intervention across multiple attributes. MAT-Steer learns steering vectors using an alignment objective that shifts the model's internal representations of undesirable outputs closer to those of desirable ones while enforcing sparsity and orthogonality among vectors for different attributes, thereby reducing inter-attribute conflicts. We evaluate MAT-Steer in two distinct settings: (i) on question answering (QA) tasks where we balance attributes like truthfulness, bias, and toxicity; (ii) on generative tasks where we simultaneously improve attributes like helpfulness, correctness, and coherence. MAT-Steer outperforms existing ITI and parameter-efficient fine-tuning approaches across both task types (e.g., 3% average accuracy gain across QA tasks and 55.82% win rate against the best ITI baseline).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
