基于多语言知识蒸馏的古希腊语句子嵌入模型
Sentence Embedding Models for Ancient Greek Using Multilingual Knowledge Distillation
AI总结：
针对古希腊语缺乏训练数据难构建高质量句子嵌入模型的问题，研究采用多语言知识蒸馏方法，借助自建的希英平行句子数据集训练BERT模型生成古希腊语句子嵌入，在多项任务上验证了效果并公开了数据集。
摘要：
目前已有针对古典语言（包括古希腊语和拉丁语）训练的上下文语言模型，用于词形还原、形态标注、词性标注、作者归属判定以及抄写错误检测等任务。然而，由于训练数据匮乏，为这些历史语言构建高质量的句子嵌入模型难度要大得多。在本研究中，我们采用多语言知识蒸馏方法训练BERT模型，以生成古希腊语文本的句子嵌入。高资源语言的主流句子嵌入方法依赖海量数据集，但我们的蒸馏方法仅需相对少量的翻译句子数据，就能让古希腊语模型继承这些高资源模型的特性。我们使用句子嵌入对齐方法构建了一个平行句子数据集，将古希腊语文档与英语译文对齐，并用该数据集训练我们的模型。我们在翻译搜索、语义相似度和语义检索任务上评估了模型性能，还探究了翻译偏差问题。我们的训练与评估数据集已在https://github.com/kevinkrahn/ancient-greek-datasets 免费公开。
英文摘要：
Contextual language models have been trained on Classical languages, including Ancient Greek and Latin, for tasks such as lemmatization, morphological tagging, part of speech tagging, authorship attribution, and detection of scribal errors. However, high-quality sentence embedding models for these historical languages are significantly more difficult to achieve due to the lack of training data. In this work, we use a multilingual knowledge distillation approach to train BERT models to produce sentence embeddings for Ancient Greek text. The state-of-the-art sentence embedding approaches for high-resource languages use massive datasets, but our distillation approach allows our Ancient Greek models to inherit the properties of these models while using a relatively small amount of translated sentence data. We build a parallel sentence dataset using a sentence-embedding alignment method to align Ancient Greek documents with English translations, and use this dataset to train our models. We evaluate our models on translation search, semantic similarity, and semantic retrieval tasks and investigate translation bias. We make our training and evaluation datasets freely available at https://github.com/kevinkrahn/ancient-greek-datasets .
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
