多类环境下上下文同义词的自监督检测：表型标注用例
Self-Supervised Detection of Contextual Synonyms in a Multi-Class Setting: Phenotype Annotation Use Case
摘要：
语境化的词嵌入是一种检测语境同义词的有力工具。然而，大多数当前最先进的（SOTA）深度学习概念提取方法仍然是有监督的，并且没有充分利用上下文的潜力。在本文中，我们提出了一种自我监督的预训练方法，该方法能够检测由浅层匹配创建的数据上训练的概念的上下文同义词。我们将我们的方法应用于稀疏多类设置（超过15，000个概念）中，从电子健康记录中提取表型信息。我们进一步研究了数据增强技术来解决类稀疏性的问题。我们的方法实现了一个新的SOTA的无监督表型概念注释的临床文本上F1和召回优于以前的SOTA的增益高达4.5和4.0的绝对点，分别。在仅用20%的标记数据进行微调后，我们的表现也优于BioBERT和ClinicalBERT。对三个ICU基准的外在评估也显示了使用我们的模型注释的表型作为特征的好处。
英文摘要：
Contextualised word embeddings is a powerful tool to detect contextual synonyms. However, most of the current state-of-the-art (SOTA) deep learning concept extraction methods remain supervised and underexploit the potential of the context. In this paper, we propose a self-supervised pre-training approach which is able to detect contextual synonyms of concepts being training on the data created by shallow matching. We apply our methodology in the sparse multi-class setting (over 15,000 concepts) to extract phenotype information from electronic health records. We further investigate data augmentation techniques to address the problem of the class sparsity. Our approach achieves a new SOTA for the unsupervised phenotype concept annotation on clinical text on F1 and Recall outperforming the previous SOTA with a gain of up to 4.5 and 4.0 absolute points, respectively. After fine-tuning with as little as 20\% of the labelled data, we also outperform BioBERT and ClinicalBERT. The extrinsic evaluation on three ICU benchmarks also shows the benefit of using the phenotypes annotated by our model as features.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
