发表机构
Gaoling School of Artificial Intelligence, Renmin University of China; Nanyang Technological University（中国人民大学高瓴人工智能学院; 南洋理工大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
提出关键路径识别（KPI）方法，通过识别具有强因果依赖的SAE特征构建关键路径，以少量特征修改实现更精确的引导，在RAG知识冲突任务中平均准确率提升18%。
AI 中文摘要
基于稀疏自编码器（SAE）的引导已广泛用于解决知识冲突，通过引导大语言模型（LLM）更忠实于上下文知识。现有方法通常执行大规模引导，即修改通过基于相关性的方法识别出的一大批SAE特征。然而，由于相关性不准确且忽略了特征间的交互，大规模引导方法无法精确识别在引导中起关键作用的特征，并引入了大量冗余特征，这些特征增加了噪声并削弱了引导效果。我们的实证研究表明，仅引导已识别特征中的一小部分即可达到相当甚至更好的性能。受此发现启发，我们提出了关键路径识别（KPI），一种新颖的方法，用于识别与上游和下游特征均具有强因果依赖关系的关键引导特征。KPI从这些特征构建关键路径，并通过更少的特征修改进行引导。通过这种方式，KPI将基于SAE的引导从数量驱动推进到质量聚焦，为更精确和可解释的模型编辑提供了视角。在具有知识冲突的检索增强生成（RAG）任务中的实验表明，与大规模引导的最佳基线相比，我们的方法平均提高了18%的准确率，有效过滤了冗余特征，减轻了副作用，并证明了关键路径在引导中的核心作用。
英文摘要
Sparse autoencoder (SAE)-based steering has been widely used to address knowledge conflicts by guiding LLMs to be more faithful to the contextual knowledge. Existing methods usually perform mass steering, which modifies a large batch of SAE features identified via correlation-based methods. However, due to the inaccurate correlation and the neglected feature interactions, mass steering methods fail to precisely identify the features that play the key roles in steering and introduce a large number of redundant ones, which add noise and weaken the steering effects. Our empirical studies reveal that steering only a small subset of the identified features can achieve comparable or even better performance. Motivated by this finding, we propose Key Path Identification (KPI), a novel method that identifies key steering features characterized by strong causal dependencies with both upstream and downstream features. From these features, KPI constructs key paths and steers through less feature modifications. In this way, KPI advances SAE-based steering from quantity-driven to quality-focused, offering a perspective for more precise and interpretable model editing. Experiments in RAG tasks with knowledge conflicts show that our method improves the accuracy by 18% on average compared to the best baseline of mass steering, effectively filtering redundant features, alleviating side effects and demonstrating the core role of key paths in steering.
