可调专家混合——DeepSeek-R1在推理阶段的行为修正
Mixture of Tunable Experts -- Behavior Modification of DeepSeek-R1 at Inference Time
AI总结：
本研究提出可调专家混合（MoTE）方法，借助功能性令牌共振成像技术定位DeepSeek-R1中与拒绝行为相关的专家，无需额外训练即可在推理时通过调控专家实现行为改变，为理解大模型内部机制提供了新路径。
摘要：
我们提出了可调专家混合（Mixture-of-Tunable-Experts，MoTE）方法，该方法扩展了大语言模型（Large Language Models，LLMs）的专家混合架构。无需额外训练，MoTE就能在推理阶段即时实现大语言模型有意义、针对性的行为改变。我们受功能磁共振成像（fMRI）启发，提出了名为“功能性令牌共振成像（functional Token Resonance Imaging，fTRI）”的技术，使用设计用于诱发特定行为的提示（例如“{时间}{地点}发生了什么？”）分析DeepSeek-R1的数字化大语言模型“大脑”，通过实验识别出与拒绝响应等行为相关的特异性专家。借助MoTE，我们能够干预并控制这类特定行为。我们关闭了排名前10位与拒绝最相关的专家（占R1的14848个路由专家的0.07%），在敏感参考提示上实现了52%的拒绝率降低，同时在MT-Bench上的性能没有下降。随机停用专家带来的行为偏移更小且噪声增加，而强制激活专家则会导致拒绝率显著升高。我们的方法在可解释性和可调控性方面与稀疏自编码器（sparse autoencoders，SAEs）有相似之处。与SAEs不同的是，MoTE不需要大量训练工作，因为在拥有大量专家的MoE架构中，专业化特性在预训练期间就已经自然形成。我们的研究结果表明，专家混合架构中的重要功能机制至少可以部分定位在少量特定专家中，而非分布在整个模型的权重里。通过调整专家子组可以触发显著的行为变化，这为理解大语言模型的内部运作机制提供了洞见。
英文摘要：
We present the Mixture-of-Tunable-Experts (MoTE), a method that extends the Mixture-of-Experts architecture of Large Language Models (LLMs). Without additional training, MoTE enables meaningful and focused behavior changes in LLMs on-the-fly during inference time. By analyzing the digital LLM brain of DeepSeek-R1 using a technique we dub 'functional Token Resonance Imaging' (fTRI) -- inspired by fMRI and using prompts designed to elicit specific behavior (e.g., 'What happened {time}{place}?') -- we empirically identify distinctive experts associated with behaviors like refusal responses. Using MoTE we are able to intervene and control such specific behavior. We switched off the top 10 most refusal-relevant experts (0.07% of R1's 14,848 routed experts), achieving a 52% refusal reduction on sensitive reference prompts without performance degradation on MT-Bench. Random expert deactivation resulted in smaller behavioral shifts with increased noise, whereas forced expert activation led to significantly higher refusal rates. Our approach shares similarities with sparse autoencoders (SAEs) in terms of explainability and steerability. Unlike SAEs, MoTE does not require large training efforts, as within MoEs with a vast number of experts, specialization already emerged naturally during pretraining. Our findings suggest that significant functional mechanisms in Mixture-of-Experts architectures can at least partially be localized in a small number of specific experts, rather than being distributed throughout the model's weights. Expert subgroups can be tuned to trigger significant behavior variations, providing insights into the inner workings of LLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
