用于特征发现和长上下文归因的轮次平均稀疏自编码器
Turn-Averaged SAEs for Feature Discovery and Long-Context Attribution
AI总结:
提出轮次平均稀疏自编码器,通过重构轮次平均激活来固定特征数量,简化长上下文下的特征归因与解释。
摘要:
稀疏自编码器(SAEs)已成为从语言模型中提取可解释特征的有用工具。然而,标准SAE架构对单个token激活进行操作,这意味着活跃特征的数量随上下文长度线性增长,使得研究长模型转录变得困难。我们引入了轮次平均SAEs,通过学习重构整个轮次的平均模型激活,用固定数量的特征表示单个人类或助手轮次。我们发现,当由LLM评判时,轮次平均特征比逐token特征更完整地描述了单个轮次的高级特征。我们还证明,轮次平均SAEs极大地简化了SAE的常见下游用途,如归因图。总的来说,轮次平均SAEs使得可解释性技术在长上下文长度下变得实用。
英文摘要:
Sparse autoencoders (SAEs) have become a useful tool for extracting interpretable features in language models. However, standard SAE architectures operate on individual token activations, meaning that the number of active features scales linearly with context length, and studying long model transcripts becomes difficult. We introduce turn-averaged SAEs, which represent a single Human or Assistant turn with a fixed number of features by learning to reconstruct the average model activation across the turn. We find that turn-averaged features describe a single turn's high-level characteristics more completely than per-token features when judged by an LLM. We also demonstrate that turn-averaged SAEs greatly simplify common downstream uses of SAEs like attribution graphs. Broadly, turn-averaged SAEs make interpretability techniques practical at long context lengths.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。