多语言情境学习中的语言引导
Language Steering for Multilingual In-Context Learning
AI总结：
本文提出语言向量用于多语言情境学习，通过调整隐藏状态偏移量提升模型对目标语言的识别能力，实验显示在19种语言和三种模型上均取得显著提升。
摘要：
如果大型语言模型在一个通用语义空间中运作，那么在不同语言之间切换只需简单的激活偏移。为此，我们以多语言情境学习为案例研究，提供英文的少样本演示，但测试查询为目标语言。我们提出语言向量，计算为平行源语言和目标语言示例在特定层的激活差异均值，并在推理时作为偏移量加到隐藏状态中，以将模型的内部表示转向目标语言。我们评估了该方法在三个多语言任务上的表现，涵盖19种语言和三种模型。实验结果表明，在所有测试任务和语言中，相对于基线方法，多语言情境学习均取得一致提升，证明简单的激活偏移足以将模型的语言模式转向目标语言，而无需任何参数更新。此外，这些向量编码了可解释的语言结构，相关语言形成紧密聚类，向量在不同任务间转移，表明语言身份在模型的激活空间中占据分离且结构化的方向。
英文摘要：
If large language models operate in a universal semantic space, then switching between languages should require only a simple activation offset. To test this, we take multilingual in-context learning as a case study, where few-shot demonstrations are provided in English but the test query is in a target language. We propose language vectors, computed as the mean activation difference between parallel source and target language examples at a particular layer, and added as an offset to hidden states at inference time to shift the model's internal representations toward the target language. We evaluate our method across three multilingual tasks spanning 19 languages and three models. Our results show consistent improvements on multilingual in-context learning over baselines across all tasks and languages tested, demonstrating that a simple activation offset is sufficient to redirect a model's language mode without any parameter updates. Beyond performance, the vectors encode interpretable linguistic structure, with closely related languages forming tight clusters and vectors transferring across tasks, suggesting that language identity occupies separable and structured directions in a model's activation space.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
