多语言大语言模型是用英语思考的吗？
Do Multilingual LLMs Think In English?
AI总结：
本研究通过logit lens探究多语言LLM的内部表征，发现其无论输入输出语言，关键决策与推理都在接近英语的表征空间进行，且英语计算的激活转向效果更优，揭示了多语言LLM受英语主导的内部机制。
摘要：
大语言模型（LLM）具备多语言能力，可解决跨多种语言的任务。但我们研究表明，当前的LLM无论输入和输出语言是什么，都会在最接近英语的表征空间中做出关键决策。我们使用logit lens（对数似然透镜）探究法语、德语、荷兰语和普通话句子的内部表征，发现LLM对于语义负载较重的词，会先生成接近英语的表征，再将其翻译为目标语言。我们进一步证实，当使用英语而非输入输出语言计算转向向量时，这些LLM中的激活转向效果更好。这表明多语言LLM的关键推理步骤是在深受英语影响的表征中完成的，而这一点对系统用户并不透明。
英文摘要：
Large language models (LLMs) have multilingual capabilities and can solve tasks across various languages. However, we show that current LLMs make key decisions in a representation space closest to English, regardless of their input and output languages. Exploring the internal representations with a logit lens for sentences in French, German, Dutch, and Mandarin, we show that the LLM first emits representations close to English for semantically-loaded words before translating them into the target language. We further show that activation steering in these LLMs is more effective when the steering vectors are computed in English rather than in the language of the inputs and outputs. This suggests that multilingual LLMs perform key reasoning steps in a representation that is heavily shaped by English in a way that is not transparent to system users.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
