操纵分词是否有助于跨语言迁移？一项针对非标准化语言词性标注的研究
Does Manipulating Tokenization Aid Cross-Lingual Transfer? A Study on POS Tagging for Non-Standardized Languages
AI总结：
本文研究非标准化语言变体的跨语言词性标注迁移，通过在七语种上微调 PLM 并操纵分词，发现源域与目标域拆分词比率差异最能预测零样本性能。
摘要：
微调预训练语言模型（PLM）面临的挑战之一是，其分词器针对预训练所用语言进行了优化，但在面对数据中此前未见的变异时却很脆弱。例如，当在一种语言上微调 PLM，并在没有标准化正字法的密切相关语言变体的数据上评估它们时，就可以观察到这一点。尽管语言相似度很高，分词却不再对应目标数据的有意义表征，导致词性标注等任务性能低下。在本研究中，我们在来自三个不同语系的七种语言上微调 PLM，并分析它们在密切相关、非标准化变体上的零样本性能。我们考虑了源数据与目标数据分词差异的不同度量，以及在微调步骤中通过操纵分词来调整这些差异的方式。总体而言，我们发现，源数据与目标数据中被拆分为子词的单词百分比之间的相似度（即拆分词比率差异）是目标数据上模型性能的最强预测因子。
英文摘要：
One of the challenges with finetuning pretrained language models (PLMs) is that their tokenizer is optimized for the language(s) it was pretrained on, but brittle when it comes to previously unseen variations in the data. This can for instance be observed when finetuning PLMs on one language and evaluating them on data in a closely related language variety with no standardized orthography. Despite the high linguistic similarity, tokenization no longer corresponds to meaningful representations of the target data, leading to low performance in, e.g., part-of-speech tagging. In this work, we finetune PLMs on seven languages from three different families and analyze their zero-shot performance on closely related, non-standardized varieties. We consider different measures for the divergence in the tokenization of the source and target data, and the way they can be adjusted by manipulating the tokenization during the finetuning step. Overall, we find that the similarity between the percentage of words that get split into subwords in the source and target data (the split word ratio difference) is the strongest predictor for model performance on target data.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
