BERTić --波斯尼亚语、克罗地亚语、黑山语和塞尔维亚语的Transformer语言模型
BERTić -- The Transformer Language Model for Bosnian, Croatian, Montenegrin and Serbian
摘要：
在本文中，我们描述了一个Transformer模型，该模型对来自克罗地亚，波斯尼亚，塞尔维亚和黑山网络域的80亿个抓取文本标记进行了预训练。我们评估的Transformer模型的词性标注，命名实体识别，地理位置预测和常识因果推理的任务，所有的任务上显示出改进的国家的最先进的模型。对于常识推理评估，我们引入了COPA-HR --将合理替代选择（COPA）数据集翻译成克罗地亚语。BERTić模型可供免费使用，并通过HuggingFace进行进一步的特定任务微调。
英文摘要：
In this paper we describe a transformer model pre-trained on 8 billion tokens of crawled text from the Croatian, Bosnian, Serbian and Montenegrin web domains. We evaluate the transformer model on the tasks of part-of-speech tagging, named-entity-recognition, geo-location prediction and commonsense causal reasoning, showing improvements on all tasks over state-of-the-art models. For commonsense reasoning evaluation, we introduce COPA-HR -- a translation of the Choice of Plausible Alternatives (COPA) dataset into Croatian. The BERTić model is made available for free usage and further task-specific fine-tuning through HuggingFace.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
