arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.08625cs.CL
收藏

多语言大语言模型是否具有专门的语言头部?

Do Multilingual LLMs have specialized language heads?

Muhammad Naufil

AI总结:

本文研究多语言大语言模型是否为每种语言具有专门的语言注意力头部,并探讨移除不需要语言头部对性能的影响。

摘要:

多语言大语言模型(LLMs)因其能够处理和生成多语言文本的能力而受到广泛关注。然而,当只对支持语言中的子集感兴趣时,在生产环境中部署这些模型可能会效率低下。已经有一些研究探讨了机器翻译模型是否具有语言特定或语言无关的头部,但截至目前为止,针对多语言LLMs的研究尚不存在,因为这些模型能够执行超越单纯翻译的多种任务。本文探讨多语言LLMs是否为每种语言具有专门的语言注意力头部,并调查是否可以移除不需要的语言头部而不影响目标语言的性能。我们的发现可能为多语言LLMs的更高效部署策略提供指导,从而在减少模型复杂性的同时保持目标语言的高准确性。

英文摘要:

Multilingual large language models (LLMs) have gained significant popularity for their ability to process and generate text across multiple languages. However, deploying these models in production can be inefficient when only a subset of the supported languages is of interest. There has been some research conducted on identifying whether machine translation models have language-specific or language-agnostic heads, however no research has been conducted for multilingual LLMs, to the best of our knowledge, that as we know are capable of performing diverse tasks beyond just translation. This paper explores whether multilingual LLMs have specialized language attention heads for each language, and investigates the possibility of removing language-specific heads for unwanted languages without degrading performance in the targeted languages. Our findings could inform more efficient deployment strategies for multilingual LLMs, enabling reduced model complexity while maintaining high accuracy for targeted languages.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑