转向新的嵌入空间：分析多语言语言模型中模型干预所引发的跨语言对齐
Steering into New Embedding Spaces: Analyzing Cross-Lingual Alignment Induced by Model Interventions in Multilingual Language Models
AI总结：
本文研究通过模型干预而非微调来增强多语言大语言模型的跨语言对齐，发现操控特定神经元可改变嵌入空间并提升跨语言检索性能，top-1准确率最高提升2倍。
摘要：
跨语言的对齐表示是多语言大语言模型（mLLMs）所期望具备的特性，因为对齐能够提升跨语言任务的性能。通常，实现对齐需要对模型进行微调，而微调计算成本高昂，并且需要大量语言数据，这些数据往往可能无法获得。一种数据高效的微调替代方案是模型干预——一种通过操控模型激活来将生成引导至期望方向的方法。我们分析了一种流行的干预方法（寻找专家）对mLLMs中跨语言表示对齐的影响。我们针对给定语言识别出需要操控的神经元，并考察了操控前后mLLMs的嵌入空间。我们表明，修改mLLM的激活会改变其嵌入空间，从而增强跨语言对齐。此外，我们表明嵌入空间的变化转化为下游检索任务性能的提升，在跨语言检索的top-1准确率上实现了最高2倍的改进。
英文摘要：
Aligned representations across languages is a desired property in multilingual large language models (mLLMs), as alignment can improve performance in cross-lingual tasks. Typically alignment requires fine-tuning a model, which is computationally expensive, and sizable language data, which often may not be available. A data-efficient alternative to fine-tuning is model interventions -- a method for manipulating model activations to steer generation into the desired direction. We analyze the effect of a popular intervention (finding experts) on the alignment of cross-lingual representations in mLLMs. We identify the neurons to manipulate for a given language and introspect the embedding space of mLLMs pre- and post-manipulation. We show that modifying the mLLM's activations changes its embedding space such that cross-lingual alignment is enhanced. Further, we show that the changes to the embedding space translate into improved downstream performance on retrieval tasks, with up to 2x improvements in top-1 accuracy on cross-lingual retrieval.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
