为何更好的跨语言对齐未能带来更好的跨语言迁移：以编码器为例
Why Better Cross-Lingual Alignment Fails for Better Cross-Lingual Transfer: Case of Encoders
AI总结：
研究发现跨语言对齐与下游任务目标存在正交性，导致对齐效果因语言和任务类型而异，提出结合对齐与任务微调的实用指南。
摘要：
更好的跨语言对齐通常被认为能带来更好的跨语言迁移。然而，显式对齐技术尽管增加嵌入相似性，却常常无法提升词级下游性能。本文表明，这种不匹配源于对齐与下游任务目标的正交性，以及不同语言和任务类型下对齐带来的下游收益差异较大。我们分析了四个在不同语言对上对齐并针对词性标注或句子分类进行微调的XLM-R编码器模型。通过嵌入距离、梯度相似性和梯度大小等表征分析，发现：(1) 嵌入距离单独不足以预测任务性能的改进或退化；(2) 对齐与任务梯度往往接近正交，表明优化一个目标可能对另一个目标贡献有限。综合来看，我们的发现解释了为何“更好的”对齐往往无法转化为“更好的”跨语言迁移。基于这些见解，我们提供了将跨语言对齐与任务特定微调结合的实用指南，强调了精心选择损失函数的重要性。
英文摘要：
Cross-lingual alignment is often assumed to improve cross-lingual transfer by bringing representations of different languages closer together. However, improvements in representational alignment do not consistently translate into better downstream performance. We investigate this disconnect using XLM-R models explicitly aligned across four language pairs with token-level, sentence-level, and masked-language-modeling objectives. We evaluate their zero-shot transfer on a token-level task (part-of-speech tagging) and a sentence-level task (sentence classification), and analyze both representational changes and the gradients induced by the alignment and downstream objectives. We find that embedding-based alignment metrics do not reliably indicate whether alignment will improve or degrade downstream performance. Moreover, alignment and downstream-task gradients are often nearly orthogonal, particularly when the alignment objective and downstream task operate at different representational levels. These findings suggest that representation alignment alone is insufficient for assessing cross-lingual transfer, and that the compatibility between alignment and downstream objectives should be considered when designing/evaluating alignment methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
