哪种LoRA？多语言指令微调中LoRA技术有效性的实证研究
Which LoRA? An Empirical Study on the Effectiveness of LoRA Techniques During Multilingual Instruction Tuning
AI总结：
通过实验比较基本LoRA与四种变体在多语言指令微调中的效果，发现复杂变体在平衡跨语言迁移与知识保留方面并无显著优势。
摘要：
我们研究了常见的LoRA变体在多语言指令微调中是否比基本LoRA更具优势。涉及LoRA及其他四种变体在两个数据集、多种目标语言上的实验表明，使用更复杂的LoRA变体相对于基本LoRA，在平衡跨语言迁移和知识保留方面并无显著优势。对隐藏嵌入的分析显示，使用不同LoRA技术微调的大型语言模型在逐层语言表示上基本相似，这表明LoRA技术的架构新颖性可能并未转化为更好的跨语言适应能力。
英文摘要：
We investigate whether commonly available LoRA variants have an advantage over basic LoRA in multilingual instruction tuning. Experiments involving LoRA and four other variants on two datasets across diverse target languages show that there is no significant advantage in using more complex LoRA variants instead of basic LoRA, with respect to balancing cross-lingual transfer and knowledge retention. An analysis of hidden embeddings reveal that layer-wise language representation remains largely similar across LLMs fine-tuned with different LoRA techniques, suggesting that architectural novelty of LoRA techniques may not translate into better cross-lingual adaptation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
