用于医疗任务的多语言BERT语言模型：领域特定适应与跨语言性评估
Multilingual BERT language model for medical tasks: Evaluation on domain-specific adaptation and cross-linguality
AI总结：
本研究探讨在领域特定语料库上进一步预训练多语言BERT对医疗NLP任务的影响，发现领域适应和跨语言迁移能显著提升低资源语言（荷兰语、罗马尼亚语、西班牙语）的临床任务性能。
摘要：
在多语言医疗保健应用中，领域特定的自然语言处理（NLP）工具的可用性有限，特别是对于低资源语言。尽管多语言双向编码器表示来自transformers（BERT）为缓解语言鸿沟提供了有希望的动机，但低资源语言的医疗NLP任务仍然研究不足。因此，本研究调查了在领域特定语料库上的进一步预训练如何影响模型在医疗任务上的性能，重点关注三种语言：荷兰语、罗马尼亚语和西班牙语。在进一步预训练方面，我们进行了四个实验来创建医疗领域模型。然后，这些模型在三个下游任务上进行了微调：荷兰语临床笔记中的自动患者筛查，罗马尼亚语和西班牙语临床笔记中的命名实体识别。结果表明，领域适应显著增强了任务性能。此外，领域的进一步区分，例如临床和一般生物医学领域，导致了多样化的性能。临床领域适应的模型优于更一般的生物医学领域适应的模型。此外，我们观察到了跨语言可迁移性的证据。此外，我们还进行了进一步的调查，以探索导致这些性能差异的潜在原因。这些发现突出了领域适应和跨语言能力在医疗NLP中的可行性。在低资源语言设置中，这些发现可以为开发多语言医疗NLP系统提供有意义的指导，以缓解训练数据的缺乏，从而提高模型性能。
英文摘要：
In multilingual healthcare applications, the availability of domain-specific natural language processing(NLP) tools is limited, especially for low-resource languages. Although multilingual bidirectional encoder representations from transformers (BERT) offers a promising motivation to mitigate the language gap, the medical NLP tasks in low-resource languages are still underexplored. Therefore, this study investigates how further pre-training on domain-specific corpora affects model performance on medical tasks, focusing on three languages: Dutch, Romanian and Spanish. In terms of further pre-training, we conducted four experiments to create medical domain models. Then, these models were fine-tuned on three downstream tasks: Automated patient screening in Dutch clinical notes, named entity recognition in Romanian and Spanish clinical notes. Results show that domain adaptation significantly enhanced task performance. Furthermore, further differentiation of domains, e.g. clinical and general biomedical domains, resulted in diverse performances. The clinical domain-adapted model outperformed the more general biomedical domain-adapted model. Moreover, we observed evidence of cross-lingual transferability. Moreover, we also conducted further investigations to explore potential reasons contributing to these performance differences. These findings highlight the feasibility of domain adaptation and cross-lingual ability in medical NLP. Within the low-resource language settings, these findings can provide meaningful guidance for developing multilingual medical NLP systems to mitigate the lack of training data and thereby improve the model performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
