arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.02315cs.CL
收藏

信念的形状:语言模型后验表示的流形、动态和干预

The Shape of Beliefs: Geometry, Dynamics, and Interventions along Representation Manifolds of Language Models' Posteriors

Raphaël Sarfati, Eric Bigelow, Daniel Wurgaft, Jack Merullo, Atticus Geiger, Owen Lewis, Tom McGrath, Ekdeep Singh Lubana

AI总结:

研究揭示了语言模型中信念的几何结构,通过干预展示了如何在流形上保持信念的一致性,表明LLMs中自然涌现出复杂结构。

摘要:

大型语言模型(LLMs)表示提示条件下的信念(答案和主张的后验),但我们缺乏对这些信念如何在表示空间中编码、如何随着新证据更新以及如何通过干预改变的机制性解释。我们研究了一个受控环境,其中Llama-3.2通过隐式推断其参数(均值和标准差)来从正态分布中生成样本,仅基于上下文中的分布样本。我们发现,这些参数的弯曲

英文摘要:

Large language models (LLMs) represent prompt-conditioned beliefs (posteriors over answers and claims), but we lack a mechanistic account of how these beliefs are encoded in representation space, how they update with new evidence, and how interventions reshape them. We study a controlled setting in which Llama-3.2 generates samples from a normal distribution by implicitly inferring its parameters (mean and standard deviation) given only samples from the distribution in context. We find representations of curved "belief manifolds" for these parameters form with sufficient in-context learning and study how the model adapts when the distribution suddenly changes. While standard linear steering often pushes the model off-manifold and induces coupled, out-of-distribution shifts, geometry and field-aware steering better preserves the intended belief family. Our work demonstrates an example of linear field probing (LFP) as a simple approach to tile the data manifold and make interventions that respect the underlying geometry. We conclude that rich structure emerges naturally in LLMs and that purely linear concept representations are often an inadequate abstraction.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑