面向印尼语金融NLP的领域特定语言模型后训练
Domain-Specific Language Model Post-Training for Indonesian Financial NLP
AI总结：
针对印尼语金融领域，对预训练IndoBERT进行小规模语料后训练，构建数据集并发布BERT模型族，验证了后训练可提升领域下游任务效果。
摘要：
BERT和IndoBERT在多项NLP任务中取得了令人瞩目的性能。已有若干研究探讨其在专业领域（尤其是英语）中的适配性。我们聚焦于金融领域和印尼语，使用小规模印尼语金融语料对预训练的IndoBERT进行金融领域的后训练。本文中，我们构建了一个印尼语自监督金融语料库、印尼语金融情感分析数据集、印尼语金融主题分类数据集，并发布了一系列用于金融NLP的BERT模型。我们还评估了领域特定后训练在情感分析和主题分类任务上的有效性。我们的研究结果表明，后训练在将语言模型微调到领域特定的下游任务时，能提升其有效性。
英文摘要：
BERT and IndoBERT have achieved impressive performance in several NLP tasks. There has been several investigation on its adaption in specialized domains especially for English language. We focus on financial domain and Indonesian language, where we perform post-training on pre-trained IndoBERT for financial domain using a small scale of Indonesian financial corpus. In this paper, we construct an Indonesian self-supervised financial corpus, Indonesian financial sentiment analysis dataset, Indonesian financial topic classification dataset, and release a family of BERT models for financial NLP. We also evaluate the effectiveness of domain-specific post-training on sentiment analysis and topic classification tasks. Our findings indicate that the post-training increases the effectiveness of a language model when it is fine-tuned to domain-specific downstream tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
