语言模型中线性真值编码的出现
Emergence of Linear Truth Encodings in Language Models
AI总结：
本研究通过透明的一层Transformer玩具模型，揭示线性真值编码在语言模型中出现的具体机制，并验证了两阶段学习动态，为理解真值表示提供了机理与实证依据。
摘要：
最近的探针研究表明，大型语言模型展现出能够区分真伪陈述的线性子空间，但其出现机制尚不清楚。我们引入了一个透明的一层Transformer玩具模型，该模型端到端地再现了这种真值子空间，并揭示了一条它们可以出现的具体路径。我们研究了一个真值编码可以出现的简单设置：一种数据分布，其中事实陈述与其他事实陈述共现（反之亦然），促使模型学习这种区分以降低未来token的语言模型损失。我们通过预训练语言模型的实验证实了这一模式。最后，在玩具设置中，我们观察到一种两阶段的学习动态：网络首先在几步内记忆单个事实关联，然后在更长的时间跨度内学习将真与假线性分离，这反过来降低了语言建模损失。综合来看，这些结果提供了线性真值表示如何在语言模型中出现的机制演示和实证动机。
英文摘要：
Recent probing studies reveal that large language models exhibit linear subspaces that separate true from false statements, yet the mechanism behind their emergence is unclear. We introduce a transparent, one-layer transformer toy model that reproduces such truth subspaces end-to-end and exposes one concrete route by which they can arise. We study one simple setting in which truth encoding can emerge: a data distribution where factual statements co-occur with other factual statements (and vice-versa), encouraging the model to learn this distinction in order to lower the LM loss on future tokens. We corroborate this pattern with experiments in pretrained language models. Finally, in the toy setting we observe a two-phase learning dynamic: networks first memorize individual factual associations in a few steps, then -- over a longer horizon -- learn to linearly separate true from false, which in turn lowers language-modeling loss. Together, these results provide both a mechanistic demonstration and an empirical motivation for how and why linear truth representations can emerge in language models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
