多语言表示的联合矩阵分解分析
A Joint Matrix Factorization Analysis of Multilingual Representations
AI总结：
提出基于联合矩阵分解的分析工具，比较多语言与单语言模型表示，发现超过33种语言中形态句法特征编码的层级差异及其与系统发生树和跨语言任务性能的关联。
摘要：
我们提出了一种基于联合矩阵分解的分析工具，用于比较多语言和单语言模型的潜在表示。作为探测方法的替代方案，该工具允许我们以联合方式分析多组表示。利用此工具，我们研究了形态句法特征在多语言预训练模型所学表示中的反映程度与方式。我们对超过33种语言和17个形态句法类别进行了大规模实证研究。研究结果表明，上下层之间在形态句法信息编码上存在差异，且类别特定差异受语言属性影响。对分解输出进行层次聚类，生成了与语言学家手工构建的系统发生树相关的树结构。此外，我们发现分解输出与不同跨语言任务中观察到的性能表现出强相关性。我们发布了代码以促进未来研究。
英文摘要：
We present an analysis tool based on joint matrix factorization for comparing latent representations of multilingual and monolingual models. An alternative to probing, this tool allows us to analyze multiple sets of representations in a joint manner. Using this tool, we study to what extent and how morphosyntactic features are reflected in the representations learned by multilingual pre-trained models. We conduct a large-scale empirical study of over 33 languages and 17 morphosyntactic categories. Our findings demonstrate variations in the encoding of morphosyntactic information across upper and lower layers, with category-specific differences influenced by language properties. Hierarchical clustering of the factorization outputs yields a tree structure that is related to phylogenetic trees manually crafted by linguists. Moreover, we find the factorization outputs exhibit strong associations with performance observed across different cross-lingual tasks. We release our code to facilitate future research.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
