语言特异性神经元不会促进跨语言迁移
Language-specific Neurons Do Not Facilitate Cross-Lingual Transfer
AI总结：
本研究针对多语言大语言模型低资源语言性能下降问题，探究语言特异性神经元识别技术结合神经元级LoRA微调能否提升跨语言任务表现，实验发现这类干预无法带来下游任务提升，为多语言LLM研究提供关键参考。
摘要：
多语言大语言模型（LLMs）旨在实现跨多种语言的鲁棒自然语言理解，但其在低资源语言上的性能会显著下降。本研究探讨了现有识别语言特异性神经元的技术能否用于提升低资源语言的跨语言任务性能。我们开展了详尽实验，覆盖现有语言特异性神经元识别技术（如语言激活概率熵、基于激活概率的阈值法），以及针对神经元的LoRA微调，采用Llama 3.1、Mistral Nemo等模型。我们发现这类神经元层面的干预措施不足以在低资源语言的下游任务（XNLI、XQuAD）上带来跨语言性能提升。本研究揭示了实现跨语言泛化面临的挑战，为多语言大语言模型研究提供了关键洞见。
英文摘要：
Multilingual large language models (LLMs) aim towards robust natural language understanding across diverse languages, yet their performance significantly degrades on low-resource languages. This work explores whether existing techniques to identify language-specific neurons can be leveraged to enhance cross-lingual task performance of lowresource languages. We conduct detailed experiments covering existing language-specific neuron identification techniques (such as Language Activation Probability Entropy and activation probability-based thresholding) and neuron-specific LoRA fine-tuning with models like Llama 3.1 and Mistral Nemo. We find that such neuron-specific interventions are insufficient to yield cross-lingual improvements on downstream tasks (XNLI, XQuAD) in lowresource languages. This study highlights the challenges in achieving cross-lingual generalization and provides critical insights for multilingual LLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
