大语言模型中的机制调节：通过稀疏自编码器检索和引导高阶语义特征
Mechanistic Knobs in LLMs: Retrieving and Steering High-Order Semantic Features via Sparse Autoencoders
AI总结：
本文提出基于稀疏自编码器的框架，通过对比特征检索管道和生成验证，实现对高阶语言行为相关语义内部特征的引导，展示了在Big Five人格特质案例中的精准双向调控效果。
摘要：
近期机械可解释性（MI）研究使能够识别和干预大型语言模型（LLMs）的内部特征成为可能。然而，将这些内部特征与语言生成中复杂行为层面的语义属性的可靠控制联系起来仍是一个持续的挑战。本文提出了一种基于稀疏自编码器的框架，用于检索和引导与高阶语言行为相关的语义可解释内部特征。我们的方法采用基于受控语义对立的对比特征检索流程，结合统计激活分析和生成验证，从稀疏激活空间中提炼出单义功能特征。使用Big Five人格特质作为案例研究，我们证明了我们的方法能够在维持优于现有激活引导方法如对比激活加法（CAA）的稳定性和性能的同时，实现精确的双向引导。我们进一步发现了一个经验效应，我们称之为功能性忠实性，即干预特定内部特征会在多个语言维度上引起一致且可预测的转变，这些转变与目标语义属性对齐。我们的发现表明，LLMs内部化了高阶概念的深度整合表示，并提供了一种新的、稳健的机械路径，用于调节复杂AI行为。
英文摘要：
Recent work in Mechanistic Interpretability (MI) has enabled the identification and intervention of internal features in Large Language Models (LLMs). However, a persistent challenge lies in linking such internal features to the reliable control of complex, behavior-level semantic attributes in language generation. In this paper, we propose a Sparse Autoencoder-based framework for retrieving and steering semantically interpretable internal features associated with high-level linguistic behaviors. Our method employs a contrastive feature retrieval pipeline based on controlled semantic oppositions, combing statistical activation analysis and generation-based validation to distill monosemantic functional features from sparse activation spaces. Using the Big Five personality traits as a case study, we demonstrate that our method enables precise, bidirectional steering of model behavior while maintaining superior stability and performance compared to existing activation steering methods like Contrastive Activation Addition (CAA). We further identify an empirical effect, which we term Functional Faithfulness, whereby intervening on a specific internal feature induces coherent and predictable shifts across multiple linguistic dimensions aligned with the target semantic attribute. Our findings suggest that LLMs internalize deeply integrated representations of high-order concepts, and provide a novel, robust mechanistic path for the regulation of complex AI behaviors.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
