通过LLM预训练追溯人格向量
Tracing Persona Vectors Through LLM Pretraining
AI总结：
研究LLM预训练中人格向量的形成机制，发现其在早期阶段就形成并持续优化，为模型安全性和可控性提供新视角。
摘要：
大型语言模型内部如何表示高层行为是人工智能安全的核心可解释性问题：它决定了我们能检测、审计或干预什么。最近的研究表明，如邪恶或阿谀奉承等特质对应于内部激活的线性方向，即所谓的性格向量。尽管这些向量现在被广泛用于检查和引导模型行为，但其在训练过程中如何形成仍不清楚。为填补这一空白，我们追踪了OLMo-3-7B的预训练过程，发现性格向量在预训练早期（在OLMo-3预训练中仅占0.22%）就形成，并在完全训练后仍能有效引导指令模型。虽然核心表示在早期形成，但性格向量在整个预训练过程中持续几何和语义上优化。我们进一步比较了替代的提取策略，发现所有策略都能产生有效的方向，每种策略揭示了底层性格的不同定性方面。在Apertus-8B上复制我们的分析表明，我们的发现超越了OLMo-3。我们的结果确立了性格表示作为早期预训练的稳定特征，并为研究训练如何形成、优化和塑造它们开辟了新路径。
英文摘要：
How large language models internally represent high-level behaviors is a core interpretability question with direct relevance to AI safety: it determines what we can detect, audit, or intervene on. Recent work has shown that traits such as evil or sycophancy correspond to linear directions in the internal activations, the so-called persona vectors. Although these vectors are now routinely utilized to inspect and steer model behavior in safety-relevant settings, how these representations are formed during training remains unknown. To address this gap, we trace persona vectors across the pretraining of OLMo-3-7B, finding that persona vectors form remarkably early -- within 0.22% of OLMo-3 pretraining -- and remain effective for steering the fully post-trained instruct models. Although core representations are formed early on, persona vectors continue to refine geometrically and semantically throughout pretraining. We further compare alternative elicitation strategies and find that all yield effective directions, with each strategy surfacing qualitatively distinct facets of the underlying persona. Replicating our analysis on Apertus-8B reveals that our findings transfer qualitatively beyond OLMo-3. Our results establish persona representations as stable features of early pretraining and open a path to studying how training forms, refines, and shapes them.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
