控制多语言大语言模型中的语言混淆
Controlling Language Confusion in Multilingual LLMs
AI总结：
本文针对多语言大语言模型的语言混淆问题，提出在标准 SFT 基础上引入 ORPO 惩罚项以抑制跨语言混合输出，实验证明该方法在低资源和高解码温度下仍能保持语言一致性并维持问答性能。
摘要：
大语言模型经常遭受语言混淆问题，即响应部分或完全以非预期语言生成。这严重损害用户体验，尤其是在低资源场景中。我们假设该问题源于传统微调目标（如监督学习）的局限性，这些目标只优化正确 token 的似然，而未明确惩罚跨语言混合等不期望的输出。对预训练期间损失轨迹的分析进一步揭示，模型无法区分单语文本和语言混合文本，表明缺乏避免此类混淆的内在压力。在本工作中，我们应用 ORPO，它在标准 SFT 的基础上为不期望的输出风格增加惩罚，从而有效抑制语言混淆的生成。ORPO 即使在高解码温度下也能保持较强的语言一致性，同时保持通用问答性能。我们的发现表明，引入适当的惩罚项可以有效缓解多语言模型中的语言混淆，尤其是在低资源场景中。
英文摘要：
Large language models often suffer from language confusion, a phenomenon in which responses are partially or entirely generated in unintended languages. This critically degrades the user experience, especially in low-resource settings. We hypothesize that this issue stems from limitations in conventional fine-tuning objectives, such as supervised learning, which optimize the likelihood of correct tokens without explicitly penalizing undesired outputs such as cross-lingual mixing. Analysis of loss trajectories during pretraining further reveals that models fail to distinguish between monolingual and language-mixed texts, highlighting the absence of inherent pressure to avoid such confusion. In this work, we apply ORPO, which adds penalties for unwanted output styles to standard SFT, effectively suppressing language-confused generations. ORPO maintains strong language consistency, even under high decoding temperatures, while preserving general QA performance. Our findings suggest that incorporating appropriate penalty terms can effectively mitigate language confusion in multilingual models, particularly in low-resource scenarios.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
