大语言模型中超越任何语言的抽象思维的涌现
The Emergence of Abstract Thought in Large Language Models Beyond Any Language
AI总结：
本研究发现大语言模型存在核心语言无关参数空间，由跨语言共享神经元构成，支撑抽象思维涌现，并提出适配不同发展阶段的神经元特异性训练策略，经多类模型实验验证有效。
摘要：
随着大语言模型（LLMs）持续发展，其在多种语言中有效运作的能力已显著提升。初步研究观察到，即便在响应非英语提示时，LLMs的隐藏激活也常与英语相似，这催生了LLMs可能用英语“思考”的普遍假设。但近期研究显示LLMs具备强大的多语言性能，甚至在其他语言的特定任务上超越英语表现，对这一观点提出了挑战。本研究发现，LLMs会逐步形成一个核心的语言无关参数空间——这是一个极小的参数子集，停用它会导致所有语言下的性能显著下降。这一紧凑却关键的参数集是模型实现跨语言泛化能力的基础，支撑了不依附于任何特定语言系统的抽象思维的涌现。具体而言，我们识别出语言相关神经元——即在处理特定语言时持续激活的神经元，并将其分为共享型（在多种语言中激活）和专属型（仅针对单一语言）两类。随着LLMs随时间不断发展，我们观察到共享神经元的占比和功能重要性均显著提升，而专属神经元的影响力则逐渐减弱。这些共享神经元构成了核心语言无关参数空间的支柱，支撑着抽象思维的涌现。基于这些发现，我们提出了针对不同发展阶段LLMs语言无关程度的神经元特异性训练策略。在多个LLM家族上开展的实验验证了我们方法的有效性。
英文摘要：
As large language models (LLMs) continue to advance, their capacity to function effectively across a diverse range of languages has shown marked improvement. Preliminary studies observe that the hidden activations of LLMs often resemble English, even when responding to non-English prompts. This has led to the widespread assumption that LLMs may "think" in English. However, more recent results showing strong multilingual performance, even surpassing English performance on specific tasks in other languages, challenge this view. In this work, we find that LLMs progressively develop a core language-agnostic parameter space-a remarkably small subset of parameters whose deactivation results in significant performance degradation across all languages. This compact yet critical set of parameters underlies the model's ability to generalize beyond individual languages, supporting the emergence of abstract thought that is not tied to any specific linguistic system. Specifically, we identify language-related neurons-those are consistently activated during the processing of particular languages, and categorize them as either shared (active across multiple languages) or exclusive (specific to one). As LLMs undergo continued development over time, we observe a marked increase in both the proportion and functional importance of shared neurons, while exclusive neurons progressively diminish in influence. These shared neurons constitute the backbone of the core language-agnostic parameter space, supporting the emergence of abstract thought. Motivated by these insights, we propose neuron-specific training strategies tailored to LLMs' language-agnostic levels at different development stages. Experiments across diverse LLM families support our approach.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
