研究提出生成因果中介（GCM）方法，通过对比长形式响应选择模型组件，以局部化和控制扩散行为，如诗歌式 vs. 论文式表达，在三种语言模型上验证其有效性。
摘要：
在语言模型（LM）中，应在哪里干预以局部化和控制跨多个token的扩散行为？我们引入生成因果中介（GCM），一种通过对比长形式响应选择模型组件（如注意力头）来引导此类扩散概念（如诗歌式 vs. 论文式表达）的程序。在GCM中，我们首先构建对比行为输入和长形式响应的数据集。然后量化模型组件如何中介概念并选择最强的中介体进行引导。我们在三种行为——拒绝、阿谀奉承和风格转换——上评估GCM，跨三种语言模型。GCM成功局部化长形式响应中的概念，并在使用稀疏注意力头引导时优于相关性探针基线。这些结果表明，GCM为从LM的长形式响应中局部化和控制提供了有效方法。
英文摘要：
Where should we intervene in a language model (LM) to localize and control behaviors that are diffused across many tokens of a long-form response? We introduce Generative Causal Mediation (GCM), a procedure for selecting model components (e.g., attention heads) from contrastive long-form responses, to steer such diffuse concepts (e.g., talk in verse vs. talk in prose). In GCM, we first construct a dataset of contrasting behavioral inputs and long-form responses. Then, we quantify how model components mediate the concept and select the strongest mediators for steering. We evaluate GCM on three behaviors--refusal, sycophancy, and style transfer--across three language models. GCM successfully localizes concepts expressed in long-form responses and outperforms correlational probe-based baselines when steering with a sparse set of attention heads. Together, these results demonstrate that GCM provides an effective approach for localizing from and controlling the long-form responses of LMs.
Where Steering Signals Come From: Activation Source Selection in Activation Steering
Peking University（北京大学）；Tsinghua University（清华大学）；Beijing Academy of Artificial Intelligence（北京智源人工智能研究院）；Tsinghua Shenzhen International Graduate School（清华大学深圳国际研究生院）；YiXin-AILab（易信人工智能实验室）
Fine-Grained Activation Steering: Steering Less, Achieving More
School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore（电子与电气工程学院，南洋理工大学，新加坡）；Home Team Science and Technology Agency (HTX), Singapore（家庭团队科技局（HTX），新加坡）
Interpretable Steering of Large Language Models with Feature Guided Activation Additions
Raffles Science Institute（莱佛士科学学院）；Raffles Institution（莱佛士书院）；Nous Research（努斯研究机构）；Centre for Frontier AI Research (CFAR)（前沿人工智能研究中心）；Agency for Science Technology and Research (A*STAR)（新加坡科技研究局）
Language Model Activations Inhabit Privileged Error-Correcting Basins
Goodfire；University of Southern California（南加州大学）；Massachusetts Institute of Technology（麻省理工学院）；Northeastern University（东北大学）；Stanford University（斯坦福大学）；Harvard University（哈佛大学）
LF-Steering: Latent Feature Activation Steering for Enhancing Semantic Consistency in Large Language Models
College of Intelligence and Computing, Tianjin University（天津大学智能与计算学部）；IT Innovation and Research Center, Huawei Technologies（华为技术有限公司 IT 创新研究中心）；Informatics, University of Edinburgh（爱丁堡大学信息学院）
From Weights to Activations: Is Steering the Next Frontier of Adaptation?
Saarland University（萨尔兰大学）；German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心）；Centre for European Research in Trusted AI (CERTAIN)（可信AI欧洲研究中心）；Munich Center for Machine Learning（慕尼黑机器学习中心）；Fraunhofer Heinrich Hertz Institute（弗劳恩霍夫海因里希·赫兹研究所）；Technological University Dublin（都柏林技术大学）；Technische Universität Berlin（柏林技术大学）；Berlin Institute for the Foundations of Learning and Data (BIFOLD)（柏林学习与数据基础研究所）
Reflection Steering: Disentangling Reflection from Reasoning in Activation Space for Token-Efficient Inference
School of Computing and Data Science, The University of Hong Kong（香港大学计算与数据科学学院）；Department of Computing, The Hong Kong Polytechnic University（香港理工大学计算学院）；School of Data Science, Lingnan University（岭南大学数据科学学院）；Centre for Learning, Teaching and Technology, The Education University of Hong Kong（香港教育大学学习、教学与技术中心）
