对齐如何增强大语言模型的多语言能力？一种语言神经元视角
How Does Alignment Enhance LLMs' Multilingual Capabilities? A Language Neurons Perspective
AI总结：
本文从语言神经元角度探讨对齐如何提升大语言模型的多语言能力，提出三类神经元分类方法，并分析多语言推理的四个阶段。
摘要：
多语言对齐是一种有效且具有代表性的范式，用于增强大语言模型（LLMs）的多语言能力，通过将高资源语言的能力转移到低资源语言。同时，一些针对语言特有神经元的研究提供了新的视角来分析和理解LLMs的机制。然而，我们发现许多神经元被多种但并非所有语言共享，无法正确分类。在本文中，我们提出了一种三元分类方法，将神经元分为三种类型：语言特有神经元、语言相关神经元和通用神经元。我们还提出了一种相应的识别算法来区分这些不同类型的神经元。此外，基于不同类型的神经元的分布特征，我们将LLMs的多语言推理内部过程分为四个部分：（1）多语言理解，（2）共享语义空间推理，（3）多语言输出空间转换，（4）词汇空间输出。此外，我们系统地分析了对齐前后的模型，重点关注不同类型的神经元。我们还分析了“自发多语言对齐”现象。总体而言，本文基于不同类型的神经元进行了全面调查，提供了实证结果和有价值的见解，以更好地理解多语言对齐和LLMs的多语言能力。
英文摘要：
Multilingual Alignment is an effective and representative paradigm to enhance LLMs' multilingual capabilities, which transfers the capabilities from the high-resource languages to the low-resource languages. Meanwhile, some research on language-specific neurons provides a new perspective to analyze and understand LLMs' mechanisms. However, we find that there are many neurons that are shared by multiple but not all languages and cannot be correctly classified. In this work, we propose a ternary classification methodology that categorizes neurons into three types, including language-specific neurons, language-related neurons, and general neurons. And we propose a corresponding identification algorithm to distinguish these different types of neurons. Furthermore, based on the distributional characteristics of different types of neurons, we divide the LLMs' internal process for multilingual inference into four parts: (1) multilingual understanding, (2) shared semantic space reasoning, (3) multilingual output space transformation, and (4) vocabulary space outputting. Additionally, we systematically analyze the models before and after alignment with a focus on different types of neurons. We also analyze the phenomenon of "Spontaneous Multilingual Alignment". Overall, our work conducts a comprehensive investigation based on different types of neurons, providing empirical results and valuable insights to better understand multilingual alignment and multilingual capabilities of LLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
