词典来救场：利用双语词典为低资源语言进行跨语言词表迁移
Dictionaries to the Rescue: Cross-Lingual Vocabulary Transfer for Low-Resource Languages Using Bilingual Dictionaries
AI总结：
本文提出利用双语词典进行跨语言词表迁移，通过逐步移除 BPE 子词并迭代估计嵌入，在低资源语言上优于现有方法。
摘要：
跨语言词表迁移在将预训练语言模型适配到新语言（包括低资源语言）方面具有广阔前景。现有利用单语或平行语料的方法在应用于资源有限的语言时面临挑战。在本工作中，我们提出了一种简单而有效的词表迁移方法，该方法利用双语词典；得益于描写语言学家的工作，许多语言都有双语词典可用。我们提出的方法利用了 BPE 分词器的一个特性：从词表中移除一个子词会导致回退到更短的子词。目标子词的嵌入通过逐步将其从分词器中移除来进行迭代估计。实验结果表明，我们的方法在低资源语言上优于现有方法，证明了基于词典的跨语言词表迁移方法的有效性。
英文摘要：
Cross-lingual vocabulary transfer plays a promising role in adapting pre-trained language models to new languages, including low-resource languages. Existing approaches that utilize monolingual or parallel corpora face challenges when applied to languages with limited resources. In this work, we propose a simple yet effective vocabulary transfer method that utilizes bilingual dictionaries, which are available for many languages, thanks to descriptive linguists. Our proposed method leverages a property of BPE tokenizers where removing a subword from the vocabulary causes a fallback to shorter subwords. The embeddings of target subwords are estimated iteratively by progressively removing them from the tokenizer. The experimental results show that our approach outperforms existing methods for low-resource languages, demonstrating the effectiveness of a dictionary-based approach for cross-lingual vocabulary transfer.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
