分词还是不分词：跨语言迁移中文本表示的比较研究
To token or not to token: A Comparative Study of Text Representations for Cross-Lingual Transfer
AI总结：
本研究比较了四种文本表示模型在跨语言迁移中的表现，提出语言商数指标，发现基于图像的模型适用于相似语言，基于分段的模型适用于词义任务，字符级模型适用于依存分析，并据此提供模型选择建议。
摘要：
选择合适的标记化方案往往是低资源跨语言迁移中的一个瓶颈。为了理解文本表示选择对下游任务的影响，我们对具有不同文本表示模态的语言模型进行了比较分析，包括2个基于分段的模型（BERT、mBERT）、1个基于图像的模型（PIXEL）和1个字符级模型（CANINE）。首先，我们提出了一种评分语言商数（LQ）指标，能够提供零样本和少样本评估的加权组合表示。利用该指标，我们在三个任务（词性标注、依存句法分析和命名实体识别）上进行了包含19种源语言和133种目标语言的实验。我们的分析表明，当语言密切相关且共享视觉上相似的文字系统时，基于图像的模型在跨语言迁移中表现出色。然而，对于偏向词义的任务（词性标注、命名实体识别），基于分段的模型被证明更优。此外，在词关系起关键作用的依存句法分析任务中，具有字符级关注点的模型优于其他模型。最后，我们基于研究结果提出了一种推荐方案，以根据任务和语言要求指导模型选择。
英文摘要：
Choosing an appropriate tokenization scheme is often a bottleneck in low-resource cross-lingual transfer. To understand the downstream implications of text representation choices, we perform a comparative analysis on language models having diverse text representation modalities including 2 segmentation-based models (\texttt{BERT}, \texttt{mBERT}), 1 image-based model (\texttt{PIXEL}), and 1 character-level model (\texttt{CANINE}). First, we propose a scoring Language Quotient (LQ) metric capable of providing a weighted representation of both zero-shot and few-shot evaluation combined. Utilizing this metric, we perform experiments comprising 19 source languages and 133 target languages on three tasks (POS tagging, Dependency parsing, and NER). Our analysis reveals that image-based models excel in cross-lingual transfer when languages are closely related and share visually similar scripts. However, for tasks biased toward word meaning (POS, NER), segmentation-based models prove to be superior. Furthermore, in dependency parsing tasks where word relationships play a crucial role, models with their character-level focus, outperform others. Finally, we propose a recommendation scheme based on our findings to guide model selection according to task and language requirements.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
