arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2104.08464cs.CL
收藏

形态句法探测的多标记方法

A multilabel approach to morphosyntactic probing

University of Washington(华盛顿大学)

Naomi Tachikawa Shapiro, Amandalynne Paullada, Shane Steinert-Threlkeld

摘要:

我们引入了一个多标签探测任务来评估多语言语言模型中词嵌入的形态句法表征。我们用多语言BERT演示了这一任务(Devlin等人,2018年),训练探针为七种不同形态复杂性的类型多样的语言:南非荷兰语,克罗地亚语,芬兰语,希伯来语,韩语,西班牙语和土耳其语。通过这个简单但强大的范例,我们表明,多语言BERT呈现许多形态句法特征容易和同时提取(例如,性别、语法格、代名词类型)。我们进一步评估探针上的六个“坚持”的语言在一个zero-shot传输设置:阿拉伯语,中文,马拉地语,斯洛文尼亚语,塔吉文,约鲁巴语。这种类型的探测还有一个额外的好处,那就是揭示了语言模型认为跨语言共享的语言属性。例如,探测器在识别被保留语言中的名词时表现良好,这表明多语言的BERT具有超越单个语言的名词概念;然而,形容词并非如此。

英文摘要:

We introduce a multilabel probing task to assess the morphosyntactic representations of word embeddings from multilingual language models. We demonstrate this task with multilingual BERT (Devlin et al., 2018), training probes for seven typologically diverse languages of varying morphological complexity: Afrikaans, Croatian, Finnish, Hebrew, Korean, Spanish, and Turkish. Through this simple but robust paradigm, we show that multilingual BERT renders many morphosyntactic features easily and simultaneously extractable (e.g., gender, grammatical case, pronominal type). We further evaluate the probes on six "held-out" languages in a zero-shot transfer setting: Arabic, Chinese, Marathi, Slovenian, Tagalog, and Yoruba. This style of probing has the added benefit of revealing the linguistic properties that language models recognize as being shared across languages. For instance, the probes performed well on recognizing nouns in the held-out languages, suggesting that multilingual BERT has a conception of noun-hood that transcends individual languages; yet, the same was not true of adjectives.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑