从FreEM到D' AlemBERT：一个大型语料库和早期现代法语的语言模型
From FreEM to D'AlemBERT: a Large Corpus and a Language Model for Early Modern French
摘要：
语言历史状态的语言模型变得越来越重要，以便对旧文本源进行最佳数字化和分析。由于这些历史状态同时处理起来更复杂，并且在可用的语料库中更稀缺，因此需要专门的努力来训练适应数据的自然语言处理（NLP）工具。在本文中，我们提出了我们的努力，以开发自然语言处理工具的早期现代法语（历史法语从16 $^\text{th}$到18 $^\text{th}$世纪）。我们介绍了早期现代法语的$\text{FreEM}_{\text{max}}$语料库和D'AlemBERT，一个在$\text{FreEM}_{\text{max}}$上训练的基于Roberta的语言模型。我们评估的有用性的D'AlemBERT微调它的词性标注任务，优于以前的工作测试集。重要的是，我们找到了语言模型迁移学习能力的证据，因为它在资源较少的时间段的表现似乎得到了资源较多的时间段的提升。我们发布了D'AlemBERT和$\text{FreEM}_{\text{max}}$语料库的开源子部分。
英文摘要：
Language models for historical states of language are becoming increasingly important to allow the optimal digitisation and analysis of old textual sources. Because these historical states are at the same time more complex to process and more scarce in the corpora available, specific efforts are necessary to train natural language processing (NLP) tools adapted to the data. In this paper, we present our efforts to develop NLP tools for Early Modern French (historical French from the 16$^\text{th}$ to the 18$^\text{th}$ centuries). We present the $\text{FreEM}_{\text{max}}$ corpus of Early Modern French and D'AlemBERT, a RoBERTa-based language model trained on $\text{FreEM}_{\text{max}}$. We evaluate the usefulness of D'AlemBERT by fine-tuning it on a part-of-speech tagging task, outperforming previous work on the test set. Importantly, we find evidence for the transfer learning capacity of the language model, since its performance on lesser-resourced time periods appears to have been boosted by the more resourced ones. We release D'AlemBERT and the open-sourced subpart of the $\text{FreEM}_{\text{max}}$ corpus.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
