arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2204.07056cs.CLstat.APstat.CO
收藏

用于临床文本数据去识别的Transformer模型的比较评估

A Comparative Evaluation Of Transformer Models For De-Identification Of Clinical Text Data

University of Toronto(多伦多大学)

Christopher Meaney, Wali Hakimpour, Sumeet Kalia, Rahim Moineddin

摘要:

目的:旨在比较评估几种Transformer模型架构在i2 b2/UTHealth 2014临床文本去识别挑战语料库中识别受保护健康信息(PHI)的能力。方法:i2 b2/UTHealth 2014语料库包含从N=296名患者中获得的N=1304份临床记录。使用迁移学习框架,我们在语料库上微调了几个Transformer模型架构,包括:BERT-base,BERT-large,ROBERTA-base,ROBERTA-large,ALBERT-base和ALBERT-xxlarge。在微调过程中,我们改变了以下模型超参数:批量大小,训练次数,学习率和权重衰减。我们在一个训练数据集上微调模型,在一个独立的验证数据集上评估和选择性能最佳的模型,最后在一个支持的测试数据集上评估泛化性能。我们从准确度、精确度(阳性预测值)、召回率(灵敏度)和F1评分(精确度和召回率的调和平均值)方面评估模型性能。我们感兴趣的是整体模型性能(已识别的PHI与未识别的PHI),以及特定于PHI的模型性能。结果如下:我们观察到,ROBERTA大型模型在识别i2 b2/UTHealth 2014语料库中的PHI方面表现最好,在heldout测试语料库中实现了>99%的总体准确率和96.7%的召回率/精确率。在许多PHI类的性能是好的,但是,准确性/精确度/召回率下降识别以下实体类:职业,组织,年龄和某些位置。结论:Transformers是一个有前途的模型类/体系结构的临床文本去识别。通过最少的超参数调谐Transformers器,研究人员/临床医生有机会获得(接近)最先进的性能。

英文摘要:

Objective: To comparatively evaluate several transformer model architectures at identifying protected health information (PHI) in the i2b2/UTHealth 2014 clinical text de-identification challenge corpus. Methods: The i2b2/UTHealth 2014 corpus contains N=1304 clinical notes obtained from N=296 patients. Using a transfer learning framework, we fine-tune several transformer model architectures on the corpus, including: BERT-base, BERT-large, ROBERTA-base, ROBERTA-large, ALBERT-base and ALBERT-xxlarge. During fine-tuning we vary the following model hyper-parameters: batch size, number training epochs, learning rate and weight decay. We fine tune models on a training data set, we evaluate and select optimally performing models on an independent validation dataset, and lastly assess generalization performance on a held-out test dataset. We assess model performance in terms of accuracy, precision (positive predictive value), recall (sensitivity) and F1 score (harmonic mean of precision and recall). We are interested in overall model performance (PHI identified vs. PHI not identified), as well as PHI-specific model performance. Results: We observe that the ROBERTA-large models perform best at identifying PHI in the i2b2/UTHealth 2014 corpus, achieving >99% overall accuracy and 96.7% recall/precision on the heldout test corpus. Performance was good across many PHI classes; however, accuracy/precision/recall decreased for identification of the following entity classes: professions, organizations, ages, and certain locations. Conclusions: Transformers are a promising model class/architecture for clinical text de-identification. With minimal hyper-parameter tuning transformers afford researchers/clinicians the opportunity to obtain (near) state-of-the-art performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑