使用Sig-Transformer编码器从瑞典处方中提取信息抽取
Information Extraction from Swedish Medical Prescriptions with Sig-Transformer Encoder
摘要：
依赖于大型预训练语言模型，例如来自Transformers的双向编码器表示（BERT）进行编码并添加简单的预测层，在许多临床自然语言处理（NLP）任务中取得了令人印象深刻的性能。在这项工作中，我们提出了一种新的扩展的Transformer架构，将签名转换与自我注意力模型。该架构被添加在嵌入层和预测层之间。一个新的瑞典处方数据上的实验表明，该架构是优于三个信息抽取任务中的两个，与基线模型。最后，我们评估了应用多语言BERT和将瑞典语文本翻译为英语然后使用临床笔记预训练的BERT模型进行编码之间的两种不同嵌入方法。
英文摘要：
Relying on large pretrained language models such as Bidirectional Encoder Representations from Transformers (BERT) for encoding and adding a simple prediction layer has led to impressive performance in many clinical natural language processing (NLP) tasks. In this work, we present a novel extension to the Transformer architecture, by incorporating signature transform with the self-attention model. This architecture is added between embedding and prediction layers. Experiments on a new Swedish prescription data show the proposed architecture to be superior in two of the three information extraction tasks, comparing to baseline models. Finally, we evaluate two different embedding approaches between applying Multilingual BERT and translating the Swedish text to English then encode with a BERT model pretrained on clinical notes.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
