检验大型语言模型中真理方向的极限
Testing the Limits of Truth Directions in LLMs
AI总结：
研究揭示了大型语言模型中真理方向的普遍性存在局限，指出其依赖于层数、任务类型和复杂性，并受指令影响显著。
摘要：
大型语言模型（LLMs）已被证明在其激活空间中沿线性真理方向编码陈述的真实性。先前研究认为这些方向在某些方面具有普遍性，但更近期的工作对这一结论提出了质疑，基于有限的跨设置泛化能力。在本工作中，我们识别出真理方向普遍性的多个限制，尚未被以前研究所理解。我们首先表明真理方向高度依赖于层数，且对普遍性的全面理解需要在模型的多个层进行探测。随后，我们表明真理方向严重依赖于任务类型，在早期层中出现于事实性任务，在后期层中出现于推理任务；它们在任务复杂性水平上也表现出不同的性能。最后，我们表明模型指令显著影响真理方向；简单的正确性评估指令显著影响真理探测器的泛化能力。我们的发现表明，关于真理方向的普遍性主张比之前所知的要有限，对于各种模型层、任务难度、任务类型和提示模板存在显著差异。
英文摘要：
Large language models (LLMs) have been shown to encode truth of statements in their activation space along a linear truth direction. Previous studies have argued that these directions are universal in certain aspects, while more recent work has questioned this conclusion drawing on limited generalization across some settings. In this work, we identify a number of limits of truth-direction universality that have not been previously understood. We first show that truth directions are highly layer-dependent, and that a full understanding of universality requires probing at many layers in the model. We then show that truth directions depend heavily on task type, emerging in earlier layers for factual and later layers for reasoning tasks; they also vary in performance across levels of task complexity. Finally, we show that model instructions can affect truth directions; simple correctness evaluation instructions significantly affect the geometry and the generalization ability of the truth probes. Our findings indicate that universality claims for truth directions are more limited than previously known, with significant differences observable for various model layers, task difficulties, task types, and prompt templates.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
