arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1904.03323cs.CL
收藏

公开可用的临床BERT嵌入

Publicly Available Clinical BERT Embeddings

Harvard-MIT(哈佛-麻省理工);MIT CSAIL(麻省理工学院计算机科学与人工智能实验室);Microsoft Research(微软研究院)

Emily Alsentzer, John R. Murphy, Willie Boag, Wei-Hung Weng, Di Jin, Tristan Naumann, Matthew B. A. McDermott

摘要:

上下文词嵌入模型如ELMo(Peters等人,2018)和BERT(Devlin等人,2018)在最近几个月大大提高了许多自然语言处理(NLP)任务的性能。然而,这些模型在专业语料库(如临床文本)上的研究很少;此外,在临床领域,还不存在公开的预训练BERT模型。在这项工作中,我们通过探索和发布临床文本的BERT模型来满足这一需求:一个用于通用临床文本,另一个用于出院总结。我们证明,使用特定领域的模型产生三个常见的临床NLP任务相比,非特异性嵌入的性能提高。这些领域特定的模型在两个临床去识别任务上的性能并不好,并认为这是去识别源文本和合成非去识别任务文本之间差异的自然结果。

英文摘要:

Contextual word embedding models such as ELMo (Peters et al., 2018) and BERT (Devlin et al., 2018) have dramatically improved performance for many natural language processing (NLP) tasks in recent months. However, these models have been minimally explored on specialty corpora, such as clinical text; moreover, in the clinical domain, no publicly-available pre-trained BERT models yet exist. In this work, we address this need by exploring and releasing BERT models for clinical text: one for generic clinical text and another for discharge summaries specifically. We demonstrate that using a domain-specific model yields performance improvements on three common clinical NLP tasks as compared to nonspecific embeddings. These domain-specific models are not as performant on two clinical de-identification tasks, and argue that this is a natural consequence of the differences between de-identified source text and synthetically non de-identified task text.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑