arXiv 2511.106582025-11-03cs.CLcs.AI

评估开放权重大型语言模型在跨多用途场景和语言的叙事性医疗报告结构化数据提取中的表现

Evaluating Open-Weight Large Language Models for Structured Data Extraction from Narrative Medical Reports Across Multiple Use Cases and Languages

Department of Radiology and Nuclear Medicine, Erasmus MC Cancer Institute, University Medical Center Rotterdam, the Netherlands（荷兰埃因霍温麦斯特大学医学中心放射科和核医学科，埃因霍温麦斯特癌症研究所）；Department of Pathology and Clinical Bioinformatics, Erasmus MC Cancer Institute, University Medical Center Rotterdam, the Netherlands（荷兰埃因霍温麦斯特大学医学中心病理学与临床生物信息学科，埃因霍温麦斯特癌症研究所）；Faculty of Informatics, Masaryk University（马萨里克大学信息学院）；Institute of Computer Science, Masaryk University（马萨里克大学计算机科学研究所）；Department of Medical Physics, Leeds Teaching Hospitals NHS Trust, UK（英国利兹教学医院国家卫生服务信托基金，利兹大学生物医学研究中心）；Leeds Biomedical Research Centre, University of Leeds, UK（利兹大学生物医学成像科，利兹，英国）；Biomedical Imaging, University of Leeds, Leeds, United Kingdom（荷兰埃因霍温麦斯特大学医学中心外科肿瘤学和胃肠外科科，埃因霍温麦斯特癌症研究所）；Department of Surgical Oncology and Gastrointestinal Surgery, Erasmus MC Cancer Institute, University Medical Center Rotterdam, the Netherlands（奥地利格拉茨BBMRI-ERIC机构）；BBMRI-ERIC, Graz, Austria（荷兰埃因霍温麦斯特大学医学中心流行病学系，罗特万，荷兰）；Department of Epidemiology, Erasmus MC University Medical Center, Rotterdam, the Netherlands（马萨里克纪念癌症研究所生物信息学研究组）；Bioinformatics Research Group, Masaryk Memorial Cancer Institute

Douwe J. Spaanderman, Karthik Prathaban, Petr Zelina, Kaouther Mouheb, Lukáš Hejtmánek, Matthew Marzetti, Antonius W. Schurink, Damian Chan, Ruben Niemantsverdr… 展开作者

Douwe J. Spaanderman, Karthik Prathaban, Petr Zelina, Kaouther Mouheb, Lukáš Hejtmánek, Matthew Marzetti, Antonius W. Schurink, Damian Chan, Ruben Niemantsverdriet, Frederik Hartmann, Zhen Qian, Maarten G. J. Thomeer, Petr Holub, Farhan Akram, Frank J. Wolters, Meike W. Vernooij, Cornelis Verhoef, Esther E. Bron, Vít Nováček, Dirk J. Grünhagen, Wiro J. Niessen, Martijn P. A. Starmans, Stefan Klein

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对现有研究局限，评估15个开放权重LLMs在三国机构六种医疗场景报告中的结构化数据提取表现，发现靠前模型接近评分者一致，中小型通用模型与大型相当，提示图和少样本策略提升约13%，为临床数据管理提供可扩展方案。

摘要：

大型语言模型（LLMs）越来越多地用于从自由文本临床记录中提取结构化信息，但此前的研究通常集中于单一任务、有限的模型和英文报告。我们在荷兰、英国和捷克共和国的三家机构，针对六种用途场景（结直肠癌肝转移、肝肿瘤、神经退行性疾病、软组织肿瘤、黑色素瘤和肉瘤）的病理和放射学报告，评估了15个开放权重LLMs。模型包括不同规模的通用型和医学专业型LLMs，并比较了六种提示策略：零样本、单样本、少样本、思维链、自一致性和提示图。使用任务适当的指标评估性能，通过共识排名聚合和线性混合效应模型量化方差。排名靠前的模型在各任务中取得了接近评分者间一致性的宏平均得分。中小型通用模型的表现与大型模型相当，而微型和专业模型的表现较差。提示图和少样本提示将性能提高了约13%。任务特定因素（包括可变复杂性和注释变异性）对结果的影响大于模型大小或提示策略。这些发现表明，开放权重LLMs可以从跨疾病、语言和机构的临床报告中提取结构化数据，为临床数据管理提供了一种可扩展的方法。

英文摘要：

Large language models (LLMs) are increasingly used to extract structured information from free-text clinical records, but prior work often focuses on single tasks, limited models, and English-language reports. We evaluated 15 open-weight LLMs on pathology and radiology reports across six use cases, colorectal liver metastases, liver tumours, neurodegenerative diseases, soft-tissue tumours, melanomas, and sarcomas, at three institutes in the Netherlands, UK, and Czech Republic. Models included general-purpose and medical-specialised LLMs of various sizes, and six prompting strategies were compared: zero-shot, one-shot, few-shot, chain-of-thought, self-consistency, and prompt graph. Performance was assessed using task-appropriate metrics, with consensus rank aggregation and linear mixed-effects models quantifying variance. Top-ranked models achieved macro-average scores close to inter-rater agreement across tasks. Small-to-medium general-purpose models performed comparably to large models, while tiny and specialised models performed worse. Prompt graph and few-shot prompting improved performance by ~13%. Task-specific factors, including variable complexity and annotation variability, influenced results more than model size or prompting strategy. These findings show that open-weight LLMs can extract structured data from clinical reports across diseases, languages, and institutions, offering a scalable approach for clinical data curation.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。