探究真理的几何结构：LLM中真理方向在逻辑转换与问答任务上的一致性与泛化性
Probing the Geometry of Truth: Consistency and Generalization of Truth Directions in LLMs Across Logical Transformations and Question Answering Tasks
AI总结：
本文探究LLM中“真理方向”的一致性与泛化性，发现能力更强的模型表现出更一致的方向，且在声明性语句上训练的真实性探针可有效泛化至逻辑转换、问答等任务，从而提升用户信任。
摘要：
大型语言模型（LLM）在包含丰富世界知识的大型数据集上进行训练。然而，它们的输出经常包含带有自信的不准确内容。早期的研究表明，LLM将真实性编码为一个独特的线性特征，称为“truth direction（真理方向）”，它可以可靠地对真实性进行分类。我们探讨了关于truth direction的几个开放性问题：(i) LLM是否普遍表现出一致的truth direction；(ii) 是否需要复杂的探测技术来识别truth direction；以及(iii) truth direction如何在不同上下文中泛化。我们的发现表明，并非所有的LLM都表现出一致的truth direction，在能力更强的模型中观察到更强的表示，特别是在逻辑否定的上下文中。此外，我们证明了在声明性原子语句上训练的真实性探针可以有效地泛化到逻辑转换、问答任务、上下文学习和外部知识源。最后，我们探讨了真实性探针在选择式问答中的实际应用，展示了它们在提升用户对LLM输出信任度方面的潜力。这些结果推进了我们对truth direction的理解，并为LLM信念的内部表示提供了新见解。我们的代码已在 https://github.com/colored-dye/truthfulness_probe_generalization 公开
英文摘要：
Large language models (LLMs) are trained on extensive datasets that encapsulate substantial world knowledge. However, their outputs often include confidently stated inaccuracies. Earlier works suggest that LLMs encode truthfulness as a distinct linear feature, termed the "truth direction", which can classify truthfulness reliably. We address several open questions about the truth direction: (i) whether LLMs universally exhibit consistent truth directions; (ii) whether sophisticated probing techniques are necessary to identify truth directions; and (iii) how the truth direction generalizes across diverse contexts. Our findings reveal that not all LLMs exhibit consistent truth directions, with stronger representations observed in more capable models, particularly in the context of logical negation. Additionally, we demonstrate that truthfulness probes trained on declarative atomic statements can generalize effectively to logical transformations, question-answering tasks, in-context learning, and external knowledge sources. Finally, we explore the practical application of truthfulness probes in selective question-answering, illustrating their potential to improve user trust in LLM outputs. These results advance our understanding of truth directions and provide new insights into the internal representations of LLM beliefs. Our code is public at https://github.com/colored-dye/truthfulness_probe_generalization
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
