零计算跨语言迁移性估计：基于类型学特征代理

中文摘要 AI 辅助

跨语言迁移描述了源语言中的知识如何使目标语言受益。对其进行定量测量需要广泛的多语言预训练，正如先前工作通过跨语言迁移矩阵所做的那样。我们探讨迁移是否可以从免费可得的类型学特征中预测，以及高资源源语言的显著性是否反映了类型学还是数据质量和数量。我们表明，类型学数据库包含关于跨语言迁移的廉价且密集的信号。我们在先前工作的24语言迁移矩阵上仅使用类型学的随机森林，在留一语言验证中取得了ρ=0.705和R²=0.49的成绩，优于非类型学对照的ρ=0.62，这验证了仅类型学预测重建昂贵测量的跨语言迁移的能力。该信号在留一文字系统和留一语系协议中依然存在，因此文字系统和语系的混淆不能解释该效应。通过将迁移分解为类型学项和资源与文字偏差项，我们发现最佳源排名对该偏差敏感。相反，类型学不受此偏差影响，这使其成为一种零计算筛选工具，用一次模型拟合替代数百次训练运行。我们的代码可在\this https URL\this处获取。

英文摘要

Cross-lingual transfer describes how knowledge in a source language benefits a target language. Measuring it quantitatively requires broad multilingual pre-training, as prior work has done with cross-lingual transfer matrices. We ask whether transfer is predictable from freely available typological features, and whether the prominence of high-resource source languages reflects typology or data quality and quantity. We show that typological databases contain cheap and dense signals about cross-lingual transfer. Our typology-only random forest on a 24-language prior-work transfer matrix scores leave-one-language-out $ρ{=}0.705$ and $R^2{=}0.49$, beating a non-typological control at $ρ{=}0.62$, which verifies the ability of typology-only predictions to reconstruct costly measured cross-lingual transfer. The signal survives leave-one-script-out and leave-one-family-out protocols, so script and family confounding do not explain the effect. By decomposing the transfer into a typology term and a resource-and-script bias term, we find the best-source ranking sensitive to this bias. In contrast, typology is not affected by this bias, which makes it a zero-compute screening tool that replaces hundreds of training runs with a model fit. Our code is available \href{https://github.com/dharmsen/typo-x-ling-transfer}{here}.

发表机构

• Eindhoven University of Technology（埃因霍温理工大学）
• SURF

机构由 AI 辅助整理，请以论文原文为准。

补充信息

备注

4 pages, NeurIPS workshop, Linguistic Principles for Foundation Models, lp4fm