多语言大语言模型中的语言手术
Language Surgery in Multilingual Large Language Models
AI总结：
本文研究了多语言大语言模型中间层的表征对齐现象，提出ITLC方法通过潜在注入实现精确的跨语言控制，有效缓解语言混淆问题并保持语义完整性。
摘要：
大型语言模型（LLMs）在跨任务和跨语言方面展现出了卓越的泛化能力，彻底改变了自然语言处理。本文研究了LLMs中自然产生的表征对齐现象，特别是在中间层，及其对解耦特定语言信息与语言无关信息的影响。我们实证确认了这种对齐的存在，分析了其与显式设计的对齐模型相比的行为，并展示了其在不降低语义的情况下进行特定语言操作的潜力。基于这些发现，我们提出了推理时语言控制（ITLC），这是一种利用潜在注入实现精确跨语言控制并缓解LLMs中语言混淆的新方法。我们的实验突出了ITLC强大的跨语言控制能力，同时保留了目标语言的语义完整性。此外，我们证明了其在缓解跨语言语言混淆问题上的有效性，该问题即使在当前大规模LLMs中依然存在，导致不一致的语言生成。这项工作推进了我们对LLMs中表征对齐的理解，并引入了一种实用的解决方案来增强其单语和跨语言性能。
英文摘要：
Large Language Models (LLMs) have demonstrated remarkable generalization capabilities across tasks and languages, revolutionizing natural language processing. This paper investigates the naturally emerging representation alignment in LLMs, particularly in the middle layers, and its implications for disentangling language-specific and language-agnostic information. We empirically confirm the existence of this alignment, analyze its behavior in comparison to explicitly designed alignment models, and demonstrate its potential for language-specific manipulation without semantic degradation. Building on these findings, we propose Inference-Time Language Control (ITLC), a novel method that leverages latent injection to enable precise cross-lingual language control and mitigate language confusion in LLMs. Our experiments highlight ITLC's strong cross-lingual control capabilities while preserving semantic integrity in target languages. Furthermore, we demonstrate its effectiveness in alleviating the cross-lingual language confusion problem, which persists even in current large-scale LLMs, leading to inconsistent language generation. This work advances our understanding of representation alignment in LLMs and introduces a practical solution for enhancing their monolingual and cross-lingual performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
