稀疏神经元在LLMs中携带强烈的问题歧义信号
Sparse Neurons Carry Strong Signals of Question Ambiguity in LLMs
AI总结：
本研究发现LLMs内部少数稀疏神经元（AENs）线性编码问题歧义信号，基于此训练的探针在歧义检测上优于基线并跨数据集泛化，且操控AENs可控制LLM从直接回答转向弃权（不执行）。
摘要：
歧义在现实世界的问题中普遍存在，然而大型语言模型（LLMs）往往以自信的答案回应，而不是寻求澄清。在这项工作中，我们表明问题歧义线性编码在LLMs的内部表示中，并且可以在神经元级别进行检测和控制。在模型的预填充阶段，我们发现少量神经元（少至一个）编码了问题歧义信息。在这些歧义编码神经元（AENs）上训练的探针在歧义检测上取得了强劲的性能，并跨数据集泛化，优于基于提示和基于表示的基线。逐层分析表明，AENs从浅层出现，暗示模型处理流程中早期就编码了歧义信号。最后，我们表明通过操控AENs，可以控制LLM的行为，从直接回答到弃权（不执行）。我们的发现揭示了LLMs形成了问题歧义的紧凑内部表示，从而实现了可解释和可控的行为。
英文摘要：
Ambiguity is pervasive in real-world questions, yet large language models (LLMs) often respond with confident answers rather than seeking clarification. In this work, we show that question ambiguity is linearly encoded in the internal representations of LLMs and can be both detected and controlled at the neuron level. During the model's pre-filling stage, we identify that a small number of neurons, as few as one, encode question ambiguity information. Probes trained on these Ambiguity-Encoding Neurons (AENs) achieve strong performance on ambiguity detection and generalize across datasets, outperforming prompting-based and representation-based baselines. Layerwise analysis reveals that AENs emerge from shallow layers, suggesting early encoding of ambiguity signals in the model's processing pipeline. Finally, we show that through manipulating AENs, we can control LLM's behavior from direct answering to abstention. Our findings reveal that LLMs form compact internal representations of question ambiguity, enabling interpretable and controllable behavior.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
