检测和控制级联线性特征中的谄媚行为
Detecting and Controlling Sycophancy with Cascading Linear Features
AI总结:
提出迭代数据生成管线,通过级联线性特征分离谄媚行为,实现更优的检测、评分和干预,性能匹配或超越基线方法。
摘要:
通过激活引导方法解释和控制模型行为需要许多对比样本对,这些样本对清晰展示期望或不期望的行为。这些数据对决定了可解释性框架能够可靠检测导致行为的模型特征的程度,从而影响将模型引导向或远离此类行为的能力。在这项工作中,我们提出了一种迭代数据生成管线,用于隔离导致行为的级联线性特征。具体而言,我们展示了如何超越简单的二元样本对,而是隔离那些特征程度随行为线性变化的样本,从而更好地解缠特征。我们专注于检测和引导远离谄媚行为——即语言模型优先考虑用户验证的倾向。我们证明了通过级联样本发现的谄媚特征形成线性可分离子空间,并且允许选择比基线方法更清晰对应期望行为的模型激活。我们还评估了它们实现检测、确定性评分和稳健引导的能力,并发现它们要么匹配要么超越LLM-as-a-judge和系统提示基线,同时提供更低的计算需求和更多的可解释性保证。代码和数据:此 https URL
英文摘要:
Interpreting and controlling model behaviors through activation steering methods requires many pairs of contrastive samples that clearly exhibit desired or undesired behavior. These data pairs determine the degree to which interpretability frameworks can reliably detect model features responsible for a behavior, and therefore the ability to steer models toward or away from such behavior. In this work, we present an iterative data generation pipeline that isolates cascading linear features responsible for a behavior. Specifically, we show how moving beyond simple binary pairs of samples, and instead isolating samples that show degrees of features that scale linearly with behavior, allows for better disentanglement of features. We focus on detecting and steering away from sycophancy -- the tendency of language models to prioritize user validation. We demonstrate that sycophancy features discovered through cascading samples form linearly separable subspaces, and allow for selection of model activations that more clearly correspond to the desired behavior than baseline approaches. We also evaluate their ability to enable detection, deterministic scoring, and robust steering, and see that they either match or outperform LLM-as-a-judge and system prompting baselines while providing lower computational demand and more interpretability guarantees. Code & Data: https://cascading-feats.github.io/
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。