利用语言神经元重叠模式促进低资源语言的跨语言迁移
Linguistic Neuron Overlap Patterns to Facilitate Cross-lingual Transfer on Low-resource Languages
AI总结：
本文提出BridgeX-ICL方法，通过识别并激活跨语言共享神经元来改进低资源语言的零样本跨语言上下文学习，实验验证了其有效性。
摘要：
当前的大语言模型（LLMs）在提升低资源语言性能方面面临重大挑战，迫切需要无需昂贵微调的数据高效方法。从语言桥梁的视角出发，我们提出了一种简单而有效的方法，名为BridgeX-ICL，以改进低资源语言的零样本跨语言上下文学习（X-ICL）。与现有工作聚焦于语言特定神经元不同，BridgeX-ICL探索共享神经元是否能够提升大语言模型的跨语言性能。我们从真实标注的MUSE双语词典中构建神经元探针数据，并据此定义语言重叠神经元的子集，以确保这些锚定神经元被充分激活。随后，我们提出了一种基于HSIC的度量，用于基于重叠神经元量化大语言模型内部的语言频谱，从而指导最优桥梁选择。在4个跨语言任务和来自7个不同语系的15个语言对上进行的实验，涵盖高-低和中等-低资源语言对，验证了BridgeX-ICL的有效性，并为大语言模型底层的多语言机制提供了实证见解。代码已公开发布于https://github.com/xuyuemei/BridgeX-ICL。
英文摘要：
The current Large Language Models (LLMs) face significant challenges in improving their performance on low-resource languages and urgently need data-efficient methods without costly fine-tuning. From the perspective of language-bridge, we propose a simple yet effective method, namely BridgeX-ICL, to improve the zero-shot Cross-lingual In-Context Learning (X-ICL) for low-resource languages. Unlike existing works focusing on language-specific neurons, BridgeX-ICL explores whether sharing neurons can improve cross-lingual performance in LLMs. We construct neuron probe data from the ground-truth MUSE bilingual dictionaries, and define a subset of language overlap neurons accordingly to ensure full activation of these anchored neurons. Subsequently, we propose an HSIC-based metric to quantify LLMs' internal linguistic spectrum based on overlapping neurons, guiding optimal bridge selection. The experiments conducted on 4 cross-lingual tasks and 15 language pairs from 7 diverse families, covering both high-low and moderate-low pairs, validate the effectiveness of BridgeX-ICL and offer empirical insights into the underlying multilingual mechanisms of LLMs. The code is publicly available at https://github.com/xuyuemei/BridgeX-ICL.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
