拉丁基底：语言模型如何表示和中介文字选择
The Latin Substrate: How Language Models Represent and Mediate Script Choice
AI总结：
通过logit透镜、表征和机制分析，发现语言模型在转写时存在一致的潜在拉丁化，且通过少量后期注意力头因果中介文字选择，非拉丁输出由紧凑可识别门控产生，拉丁输出则来自网络扩散贡献，表明模型围绕共享潜在表示组织文字变异并偏向拉丁基底。
摘要：
许多语言使用多种文字书写，要求大型语言模型（LLM）以不同的正字法形式生成等价的语言内容。虽然先前的研究表明LLM通过共享的潜在表示路由信息，但它们如何内部中介文字变异仍知之甚少。我们通过首先使用logit透镜检查逐层输出分布来研究这个问题，这揭示了转写过程中一致的潜在拉丁化，然后通过文字生成的表征和机制分析。在表征层面，我们展示了同一语言的文字在层间变得越来越可分离，并且一个简单的线性引导方向可以翻转模型的输出文字，同时大致保持语义内容。该向量非对称地泛化到构建时未见过的书写系统，可靠地将非拉丁输出翻转为拉丁，但将拉丁输出映射到各种非拉丁文字。在机制层面，我们定位了一小组后期注意力头，它们因果中介文字选择。这些头跨不相关语言和书写系统转移，表明文字路由由语言无关组件实现。在这两项分析中，我们观察到一致的方向性不对称：非拉丁输出由紧凑、可识别的门控产生，而拉丁文字输出来自网络中的扩散贡献。总的来说，我们的发现暗示LLM围绕共享潜在表示组织文字变异，同时表现出对拉丁文字的优先基底。
英文摘要：
Many languages are written in multiple scripts, requiring large language models (LLMs) to generate equivalent linguistic content in distinct orthographic forms. While prior work suggests that LLMs route information through shared latent representations, how they internally mediate script variation remains poorly understood. We study this question by first examining per-layer output distributions with the logit lens, which reveals consistent latent romanization during transliteration, and then through representational and mechanistic analyses of script generation. At the representational level, we show that scripts of the same language become increasingly separable across layers and that a simple linear steering direction can flip a model's output script while largely preserving semantic content. The vector generalizes to writing systems unseen during construction: it flips non-Latin output to Latin, but maps Latin output into varied non-Latin scripts. At the mechanistic level, we localize a small set of late-layer attention heads that causally mediate script choice. These heads transfer across unrelated languages and writing systems, suggesting that script routing is implemented by language-agnostic components. Across both analyses, we observe a consistent directional asymmetry: non-Latin output is produced by a compact, identifiable gate, while Latin-script output emerges from diffuse contributions across the model. Overall, our findings hint that LLMs organize script variation around shared latent representations while exhibiting a privileged substrate toward Latin script.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
