可读但不可控：医学大语言模型幻觉的神经元级证据
Readable but Not Controllable: Neuron-Level Evidence for Medical LLM Hallucination
AI总结：
本研究通过探测医学大语言模型内部表示，发现幻觉可被高精度检测（AUROC 0.77-0.86），但该表示分布冗余且因果不可控，揭示了解码性与可控性之间的显著差距。
摘要：
幻觉仍然是部署医学大语言模型的主要障碍之一。然而，即使可以检测到幻觉，仍不清楚与之相关的内部表示是否可用于控制而不仅仅是检测。使用四个开源模型在一组医学问答数据集上，我们表明一个简单、精心条件的探针可以可靠地检测幻觉，在我们的案例中AUROC分数在0.77到0.86之间。我们进一步表明，该信号是分布且冗余的，而非狭窄局部化的。系统选择的神经元仅在非常小的子集大小上优于随机神经元，而几百个神经元的随机子集几乎恢复了全部信号，低维随机投影保留了大部分检测性能。除了检测，我们测试了这种表示是否因果可操作。在16个模型-数据集组合中，我们的结果揭示了可解码性与可控性之间的尖锐差距。使幻觉易于检测的相同内部结构并不能转化为可靠的神经元级控制。这些发现表明，医学幻觉似乎在内部激活中容易可见，但不容易通过引导与之最相关的神经元来纠正。更广泛地说，我们的结果表明，幻觉缓解不仅仅是识别正确神经元的问题，并指出了表示所揭示的内容与它们允许我们改变的内容之间的更深层次分离。
英文摘要：
Hallucination remains one of the central obstacles to deploying medical LLMs. Yet, even when hallucination can be detected, it is still unclear whether the internal representations associated with it can be used for control rather than detection alone. Using four open-source models across a suite of medical question-answering datasets, we show that a simple, carefully conditioned probe can reliably detect hallucination, with AUROC scores between 0.77 and 0.86 in our case. We further show that this signal is distributed and redundant rather than narrowly localized. Systematically selected neurons outperform random neurons only at very small subset sizes, whereas random subsets of a few hundred neurons recover nearly the full signal, and low-dimensional random projections preserve most of the detection performance. Beyond detection, we test whether this representation is causally actionable. Across 16 model--dataset combinations, our results reveal a sharp gap between decodability and controllability. The same internal structure that makes hallucination easy to detect does not translate into reliable neuron-level control. These findings show that medical hallucination seems to be readily visible in internal activations, but not easily corrected by steering the neurons most associated with it. More broadly, our results suggest that hallucination mitigation is not simply a matter of identifying the right neurons, and point to a deeper separation between what representations reveal and what they allow us to change.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
