代码切换揭示多语言大模型中的语言锚定
Code-Switching Reveals Language Anchoring in Multilingual LLMs
AI总结：
通过语法强制代码切换诊断多语言大模型中的语言锚定现象，提出锚定偏差度量并设计CANVAS干预方法，有效缓解代码切换导致的问答性能下降。
摘要：
多语言大模型（MLLMs）越来越需要处理代码切换（CS）输入，然而混合语言通常会导致性能相对于源语言或目标语言单语版本下降。为了理解这种退化，我们使用语法强制CS作为受控诊断设置，将CS表示相对于其源和目标对应物进行定位。我们引入锚定偏差（Anchor Bias），一种几何度量，用于量化语言锚定，即CS隐藏状态是否更接近其源语言或目标语言对应物。在不同的MLLMs中，锚定偏差揭示了一致的语法框架效应：源框架CS保持源锚定，而目标框架CS向目标方向移动，并显示出更大的问答（QA）退化。受这种表示模式的启发，我们提出了CANVAS（基于上下文锚定的神经向量对齐引导），一种推理时干预方法，从输入中提取源侧画布，并在预填充期间将目标语言隐藏状态软引导向源锚定。CANVAS在MLLMs和CS条件下一致地恢复了QA F1分数，表明内部锚定信号为缓解CS推理失败提供了可行的目标。
英文摘要：
Multilingual Large Language Models (MLLMs) are increasingly expected to handle Code-Switched (CS) inputs, yet mixing languages frequently degrades performance relative to source- or target-language monolingual counterparts. To understand this degradation, we use grammar-forced CS as a controlled diagnostic setting for locating CS representations relative to their source and target counterparts. We introduce Anchor Bias, a geometric measure that quantifies language anchoring, whether a CS hidden state aligns closer to its source or target language counterpart. Across diverse MLLMs, Anchor Bias reveals a consistent grammar-frame effect: source-framed CS stays source-anchored, whereas target-framed CS shifts target-ward and shows larger Question Answering (QA) degradation. Motivated by this representational pattern, we propose CANVAS (Contextual Anchor-based Neural Vector Alignment Steering), an inference-time intervention that extracts a source-side canvas from the input and softly steers target-language hidden states toward the source anchor during prefill. CANVAS consistently recovers QA F1 across MLLMs and CS conditions, showing that internal anchoring signals provide an actionable target for mitigating CS inference failures.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
