迁移神经元假说：多语言大语言模型中语言潜在空间转换的底层机制
The Transfer Neurons Hypothesis: An Underlying Mechanism for Language Latent Space Transitions in Multilingual LLMs
AI总结：
本文提出并验证迁移神经元假说，揭示MLP中特定神经元在多语言LLM里负责语言特定潜在空间与共享语义空间之间的表示迁移，并证明其对多语言推理的关键作用。
摘要：
近期研究提出了一种基于解码器的大语言模型处理多语言输入的框架：早期层将输入转换为以英语为中心且语言无关的表示；中间层在以英语为中心的潜在空间中进行推理；最终层通过将这些表示转换回语言特定的潜在空间来生成输出。然而，这种转换的内部动态及其底层机制仍未得到充分探索。为了更深入地理解这一框架，我们提出并通过实验验证了迁移神经元假说：MLP模块中的某些神经元负责在语言特定的潜在空间与共享的语义潜在空间之间迁移表示。此外，我们表明，近期研究中识别出的语言特定神经元的一个功能是促进潜在空间之间的移动。最后，我们证明迁移神经元对于多语言大语言模型的推理至关重要。
英文摘要：
Recent studies have suggested a processing framework for multilingual inputs in decoder-based LLMs: early layers convert inputs into English-centric and language-agnostic representations; middle layers perform reasoning within an English-centric latent space; and final layers generate outputs by transforming these representations back into language-specific latent spaces. However, the internal dynamics of such transformation and the underlying mechanism remain underexplored. Towards a deeper understanding of this framework, we propose and empirically validate The Transfer Neurons Hypothesis: certain neurons in the MLP module are responsible for transferring representations between language-specific latent spaces and a shared semantic latent space. Furthermore, we show that one function of language-specific neurons, as identified in recent studies, is to facilitate movement between latent spaces. Finally, we show that transfer neurons are critical for reasoning in multilingual LLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
