国家数字图书馆的运作：挪威Transformer模型案例
Operationalizing a National Digital Library: The Case for a Norwegian Transformer Model
摘要：
在这项工作中，我们展示了在国家图书馆从数字和数字化馆藏中构建大规模培训集的过程。由此产生的双向编码器表示从Transformers（BERT）为基础的语言模型挪威语优于多语言BERT（mBERT）模型在几个令牌和序列分类任务的挪威博克莫尔语和挪威语Nynorsk。我们的模型还提高了语料库中其他语言（如英语，瑞典语和丹麦语）的mBERT性能。对于语料库中未包含的语言，权重适度降低，同时保持强大的多语言属性。因此，我们表明，在内存机构内使用有点嘈杂的光学字符识别（OCR）内容构建高质量的模型是可行的，我们希望为其他内存机构铺平道路。
英文摘要：
In this work, we show the process of building a large-scale training set from digital and digitized collections at a national library. The resulting Bidirectional Encoder Representations from Transformers (BERT)-based language model for Norwegian outperforms multilingual BERT (mBERT) models in several token and sequence classification tasks for both Norwegian Bokmål and Norwegian Nynorsk. Our model also improves the mBERT performance for other languages present in the corpus such as English, Swedish, and Danish. For languages not included in the corpus, the weights degrade moderately while keeping strong multilingual properties. Therefore, we show that building high-quality models within a memory institution using somewhat noisy optical character recognition (OCR) content is feasible, and we hope to pave the way for other memory institutions to follow.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
