用于增强语言模型思维链推理的特征提取与引导
Feature Extraction and Steering for Enhanced Chain-of-Thought Reasoning in Language Models
AI总结：
受DeepSeek-R1启发，本研究提出基于稀疏自编码器及无需SAE的引导算法，直接从残差激活计算引导方向，在不依赖外部数据集微调的情况下显著增强了LLM的思维链推理能力。
摘要：
大型语言模型（LLMs）展现出利用思维链（CoT）技术解决推理和数学问题的能力。如 DeepSeek-R1 等模型所示，扩展 CoT 长度能显著增强处理复杂问题的推理能力，但这需要昂贵且高质量的 CoT 长数据与微调。受 DeepSeek-R1 深度思考范式启发，本研究利用引导技术在不依赖外部数据集的情况下增强 LLM 的推理能力。该方法首先采用稀疏自编码器（SAEs）从原始 CoT 中提取可解释特征，随后在生成过程中利用这些特征引导 LLM 的内部状态。鉴于许多 LLM 没有对应的预训练 SAEs，我们进一步提出一种无需 SAE 的引导算法，直接从 LLM 的残差激活中计算引导方向，从而免除了对显式 SAE 的需求。实验结果表明，我们基于 SAE 及后续无需 SAE 的引导算法均显著增强了 LLMs 的推理能力。
英文摘要：
Large Language Models (LLMs) demonstrate the ability to solve reasoning and mathematical problems using the Chain-of-Thought (CoT) technique. Expanding CoT length, as seen in models such as DeepSeek-R1, significantly enhances this reasoning for complex problems, but requires costly and high-quality long CoT data and fine-tuning. This work, inspired by the deep thinking paradigm of DeepSeek-R1, utilizes a steering technique to enhance the reasoning ability of an LLM without external datasets. Our method first employs Sparse Autoencoders (SAEs) to extract interpretable features from vanilla CoT. These features are then used to steer the LLM's internal states during generation. Recognizing that many LLMs do not have corresponding pre-trained SAEs, we further introduce a novel SAE-free steering algorithm, which directly computes steering directions from the residual activations of an LLM, obviating the need for an explicit SAE. Experimental results demonstrate that both our SAE-based and subsequent SAE-free steering algorithms significantly enhance the reasoning capabilities of LLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
