真理的几何：大语言模型真假数据集表征中的涌现线性结构
The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets
AI总结：
该研究通过可视化、迁移实验和因果干预三方面证据，验证了大语言模型对事实真假存在线性表征结构，且简单均值差探针泛化性佳且因果关联性更强。
摘要：
大语言模型（LLMs）能力出众，但容易输出错误信息。近期研究已开发出通过在LLM内部激活上训练探针来推断LLM是否在说真话的技术。然而，这一研究方向存在争议，部分研究者指出这些探针在基础泛化能力存在缺陷，还存在其他概念性问题。在本研究中，我们使用高质量的简单真假陈述数据集，详细研究LLM对真理的表征结构，依据三方面证据：1. LLM真假陈述表征的可视化结果，其揭示了清晰的线性结构。2. 迁移实验：在一个数据集上训练的探针可泛化到不同数据集。3. 通过对LLM前向传播进行外科手术式干预获得的因果证据，干预可使模型将假陈述视为真，反之亦然。总体而言，我们提出的证据表明，在足够规模的LLM会以线性方式表征事实陈述的真假。我们还表明，简单的均值差探针与其他探针技术的泛化能力相当，同时能识别出与模型输出因果关联更强的方向。
英文摘要：
Large Language Models (LLMs) have impressive capabilities, but are prone to outputting falsehoods. Recent work has developed techniques for inferring whether a LLM is telling the truth by training probes on the LLM's internal activations. However, this line of work is controversial, with some authors pointing out failures of these probes to generalize in basic ways, among other conceptual issues. In this work, we use high-quality datasets of simple true/false statements to study in detail the structure of LLM representations of truth, drawing on three lines of evidence: 1. Visualizations of LLM true/false statement representations, which reveal clear linear structure. 2. Transfer experiments in which probes trained on one dataset generalize to different datasets. 3. Causal evidence obtained by surgically intervening in a LLM's forward pass, causing it to treat false statements as true and vice versa. Overall, we present evidence that at sufficient scale, LLMs linearly represent the truth or falsehood of factual statements. We also show that simple difference-in-mean probes generalize as well as other probing techniques while identifying directions which are more causally implicated in model outputs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
