当对齐有害时：多语言模型中表征空间的解耦
When Alignment Hurts: Decoupling Representational Spaces in Multilingual Models
AI总结：
本文揭示多语言模型中高资源标准语（如MSA）的表征纠缠会损害低资源变体生成，并提出在线变分探测框架实现子空间解耦，在25种阿拉伯语方言上平均提升+2.0 chrF++。
摘要：
通常认为，与高资源标准语言对齐有助于对相关的低资源变体进行建模。我们通过证明与主导变体（例如现代标准阿拉伯语（MSA）相对于阿拉伯语方言）的过度表征纠缠会主动阻碍生成建模，来挑战这一假设。我们通过分析并直接干预大型语言模型（LLM）的内部表征几何，提出了对这一现象的首个全面因果研究。我们的关键贡献是一个在线变分探测框架，该框架在微调过程中持续估计标准变体的子空间，从而能够通过投影实现与该空间的解耦。虽然我们的研究以阿拉伯语为例，因为它在25种方言中拥有异常丰富的平行资源，但更广泛的动机是方法论上的：方言机器翻译（dialectal MT）可作为生成任务的受控代理，在这些任务中无法获得可比较的多变体语料库。在25种方言中，与标准微调相比，我们的干预将生成质量最多提高了+4.9 chrF++，平均提高了+2.0，尽管在标准语言性能上存在可测量的权衡。这些结果提供了因果证据，表明高资源变体的子空间主导地位会限制相关变体的生成能力。更一般地，我们将几何探测与信息论探测同子空间级因果干预统一起来，为改进密切相关语系中的生成建模提供了实用工具，并且更广泛地，为控制多语言和多领域LLM中的表征分配提供了实用工具。代码将发布。
英文摘要：
Alignment with high-resource standard languages is often assumed to aid the modeling of related low-resource varieties. We challenge this assumption by demonstrating that excessive representational entanglement with a dominant variety, such as Modern Standard Arabic (MSA) in relation to Arabic dialects, can actively hinder generative modeling. We present the first comprehensive causal study of this phenomenon by analyzing and directly intervening in the internal representation geometry of large language models (LLMs). Our key contribution is an online variational probing framework that continuously estimates the subspace of the standard variety during fine-tuning, enabling projection-based decoupling from this space. While our study uses Arabic as a case due to its unusually rich parallel resources across 25 dialects, the broader motivation is methodological: dialectal MT serves as a controlled proxy for generative tasks where comparable multi-variety corpora are unavailable. Across 25 dialects, our intervention improves generation quality by up to +4.9 chrF++ and +2.0 on average compared to standard fine-tuning, despite a measured tradeoff in standard-language performance. These results provide causal evidence that subspace dominance by high-resource varieties can restrict generative capacity for related varieties. More generally, we unify geometric and information-theoretic probing with subspace-level causal interventions, offering practical tools for improving generative modeling in closely related language families and, more broadly, for controlling representational allocation in multilingual and multi-domain LLMs. Code will be released.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
