发表机构
VISTEC; Chulalongkorn University; AI Singapore（VISTEC; 朱拉隆功大学; AI新加坡）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
提出跨语言无监督自举方法增强预训练语言模型句法知识，显著提升低资源语言零样本依存句法分析性能，并增强句法结构识别鲁棒性。
AI 中文摘要
基于编码器架构的预训练语言模型（PLMs）在各种语言理解任务的零样本跨语言迁移中展现出令人印象深刻的能力。然而，由于依存句法分析具有句法特性，将该技术应用于此任务仍面临重大挑战。为提升模型跨语言类型学的泛化能力，我们提出一种跨语言无监督自举方法，以增强预训练语言模型内部的句法知识。实验表明，我们的方法在低资源语言的零样本句法分析性能上取得了显著提升。对这些自举模型的分析揭示了其在识别句法结构方面鲁棒性的增强，这通过无参数树探针测试中更高的分数得以证明。
英文摘要
Pre-trained language models (PLMs) with encoder-based architectures have shown impressive capabilities in zero-shot cross-lingual transfer for various language understanding tasks. However, applying this technique to dependency parsing remains a significant challenge due to its syntactic nature. To boost model generalizability across linguistic typologies, we propose a cross-lingual unsupervised bootstrapping method to improve syntactic knowledge within the PLM. We show that our method achieves a significant improvement in zero-shot parsing performance in low-resource languages. Analysis of these bootstrapped models uncovers increased robustness in recognizing syntactic structures, evidenced by higher scores in parameter-free tree probing tests.
Comments11 pages, 4 figures
