Tomato, Tomahto, Tomate：多语言语言模型是否基于子词级语义概念进行理解？
Tomato, Tomahto, Tomate: Do Multilingual Language Models Understand Based on Subword-Level Semantic Concepts?
AI总结：
本研究探究多语言语言模型（mLMs）是否基于子词级语义概念理解文本，通过合并语义相似子词形成语义标记并评估，发现共享语义可提升模型预测性能，零样本结果更优，为跨语言迁移提供锚点。
摘要：
人类对文本的理解依赖于单词的通用语义概念而非其表面形式。我们的人类直觉在多大程度上适用于语言模型？在本研究中，我们探究当前多语言语言模型（mLMs）基于子词级语义概念进行理解的程度。为此，我们通过合并语义相似的子词及其嵌入形成“语义标记”，并在五个异质多语言下游任务上评估更新后的mLMs。结果显示，通用共享语义能帮助不同分词器和模型规模的mLMs在预测上取得显著进展。对分组子词的检查表明，它们表现出广泛的语义相似性，包括跨多种语言和文字的同义词与翻译词。最后，我们发现使用语义标记的零样本结果在某些分类任务上与原始模型相当甚至更优，这表明共享的子词级语义可能作为跨语言迁移的锚点。
英文摘要：
Human understanding of text depends on general semantic concepts of words rather than their superficial forms. To what extent does our human intuition transfer to language models? In this work, we study the degree to which current multilingual language models (mLMs) understand based on subword-level semantic concepts. To this end, we form "semantic tokens" by merging the semantically similar subwords and their embeddings, and evaluate the updated mLMs on five heterogeneous multilingual downstream tasks. Results show that the general shared semantics could get the models a long way in making the predictions on mLMs with different tokenizers and model sizes. Inspections of the grouped subwords show that they exhibit a wide range of semantic similarities, including synonyms and translations across many languages and scripts. Lastly, we find that the zero-shot results with semantic tokens are on par with or even better than the original models on certain classification tasks, suggesting that the shared subword-level semantics may serve as the anchors for cross-lingual transfer.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
