共享至关重要：分析LLMs中跨语言和任务的神经元
Sharing Matters: Analysing Neurons Across Languages and Tasks in LLMs
AI总结：
本研究通过将神经元分为四类，在多语LLMs中分析跨语言和任务的共享模式，发现全共享神经元对性能至关重要，且激活模式高度敏感，为多语模型研究提供洞见。
摘要：
大型语言模型（LLMs）彻底改变了自然语言处理（NLP）领域，近期的研究旨在理解其底层机制。然而，这类研究大多在单语环境下进行，主要聚焦于英语。很少有研究尝试探索LLMs在多语环境中的内部运作。在本研究中，我们旨在通过考察神经元激活如何在任务和语言之间共享来填补这一研究空白。我们根据神经元对不同语言下特定输入的反应，将其分为四个不同的类别：全共享、部分共享、特定和非激活。基于这一分类，我们使用多个LLMs在九种语言上对三个任务进行了广泛的实验，并在本工作中呈现了深入的分析。我们的发现揭示：（i）停用全共享神经元会显著降低性能；（ii）共享神经元在生成响应中起着至关重要的作用，尤其是全共享神经元；（iii）神经元激活模式高度敏感，并随任务、LLMs和语言的不同而变化。这些发现揭示了多语LLMs的内部运作，并为未来研究铺平了道路。我们发布了代码以促进该领域的研究。
英文摘要：
Large language models (LLMs) have revolutionized the field of natural language processing (NLP), and recent studies have aimed to understand their underlying mechanisms. However, most of this research is conducted within a monolingual setting, primarily focusing on English. Few studies have attempted to explore the internal workings of LLMs in multilingual settings. In this study, we aim to fill this research gap by examining how neuron activation is shared across tasks and languages. We classify neurons into four distinct categories based on their responses to a specific input across different languages: all-shared, partial-shared, specific, and non-activated. Building upon this categorisation, we conduct extensive experiments on three tasks across nine languages using several LLMs and present an in-depth analysis in this work. Our findings reveal that: (i) deactivating the all-shared neurons significantly decreases performance; (ii) the shared neurons play a vital role in generating responses, especially for the all-shared neurons; (iii) neuron activation patterns are highly sensitive and vary across tasks, LLMs, and languages. These findings shed light on the internal workings of multilingual LLMs and pave the way for future research. We release the code to foster research in this area.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
