多语言限制与词义消歧的多语言惩罚
Capacity Constraints and the Multilingual Penalty for Lexical Disambiguation
AI总结：
本研究发现多语言语言模型在词义消歧任务中面临容量限制，包括表示性、注意性和词汇相关性问题，这些因素影响了模型性能。
摘要：
多语言语言模型（LMs）有时在性能上不如单语言模型，这可能是由于容量限制所致。我们通过控制的人类相关性判断数据集，量化了这种「多语言惩罚」——这一任务需要精确的语义表示和上下文化机制——在英语和西班牙语中模糊词的处理。通过比较同一家族中的单语言和多语言LMs，我们发现多语言LMs的性能普遍降低。我们随后探讨了三种潜在的容量限制：表示性（减少的嵌入等距性）、注意性（减少对消歧线索的关注）和词汇相关性（增加的多词分隔）。多语言LMs在这些方面都显示出一些证据；此外，这些因素在统计上解释了以前归因于模型多语言状态的方差。这些发现表明，多语言LMs确实受到多种容量限制的影响，并且这些限制与消歧性能下降相关。
英文摘要：
Multilingual language models (LMs) sometimes under-perform their monolingual counterparts, possibly due to capacity limitations. We quantify this ``multilingual penalty'' for lexical disambiguation--a task requiring precise semantic representations and contextualization mechanisms--using controlled datasets of human relatedness judgments for ambiguous words in both English and Spanish. Comparing monolingual and multilingual LMs from the same families, we find consistently reduced performance in multilingual LMs. We then explore three potential capacity constraints: representational (reduced embedding isotropy), attentional (reduced attention to disambiguating cues), and vocabulary-related (increased multi-token segmentation). Multilingual LMs show some evidence of all three limitations; moreover, these factors statistically account for the variance formerly attributed to a model's multilingual status. These findings suggest both that multilingual LMs do suffer from multiple capacity constraints, and that these constraints correlate with reduced disambiguation performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
