arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.20147cs.CLcs.AI
收藏

非英语语言中小型且类别不平衡数据集的放射学文本分类

Classification of Radiological Text in Small and Imbalanced Datasets in a Non-English Language

Rigshospitalet(哥本哈根大学医院( Rigshospitalet));Medical University of Innsbruck(因斯布鲁克医科大学);Technical University of Denmark(丹麦技术大学);University of Copenhagen(哥本哈根大学)

Vincent Beliveau, Helene Kaas, Martin Prener, Claes N. Ladefoged, Desmond Elliott, Gitte M. Knudsen, Lars H. Pinborg, Melanie Ganz

AI总结:

针对丹麦语放射学报告小样本不平衡数据,比较类BERT、SetFit和LLM模型,发现领域预训练的类BERT表现最佳,但均不足以实现无监督分类,仅可用于辅助数据过滤以减少人工标注。

摘要:

医学领域的自然语言处理(NLP)在涉及非英语语言、标注样本少且类别不平衡的小型数据集的现实应用中可能表现不佳。目前对于如何处理这一问题尚无共识。我们评估了一组NLP模型,包括类BERT变换器、基于句向量的少样本学习(SetFit)以及提示式大语言模型(LLM),使用了三个来自丹麦语(一种低资源语言)癫痫患者磁共振图像放射学报告的英文数据集。我们的结果表明,在放射学报告目标领域预训练的类BERT模型目前在此场景下提供了最优性能。值得注意的是,SetFit和LLM模型的表现不如类BERT模型,其中LLM表现最差。重要的是,所研究的模型均未达到足够准确,无法在没有任何监督的情况下进行文本分类。然而,它们显示出用于数据过滤的潜力,这可以减少所需的人工标注量。

英文摘要:

Natural language processing (NLP) in the medical domain can underperform in real-world applications involving small datasets in a non-English language with few labeled samples and imbalanced classes. There is yet no consensus on how to approach this problem. We evaluated a set of NLP models including BERT-like transformers, few-shot learning with sentence transformers (SetFit), and prompted large language models (LLM), using three datasets of radiology reports on magnetic resonance images of epilepsy patients in Danish, a low-resource language. Our results indicate that BERT-like models pretrained in the target domain of radiology reports currently offer the optimal performances for this scenario. Notably, the SetFit and LLM models underperformed compared to BERT-like models, with LLM performing the worst. Importantly, none of the models investigated was sufficiently accurate to allow for text classification without any supervision. However, they show potential for data filtering, which could reduce the amount of manual labeling required.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑