解锁公共目录：针对德国肿瘤诊断的ICD编码对LLM进行指令微调
Unlocking Public Catalogues: Instruction-Tuning LLMs for ICD Coding of German Tumor Diagnoses
AI总结：
本研究通过公共目录构建指令数据集微调开放权重LLM，显著提升德国肿瘤诊断ICD编码准确率，并公开数据集与模型。
摘要：
在德国，使用ICD-10-GM和ICD-O-3对肿瘤诊断进行准确编码对于结构化的癌症文档记录至关重要。较小的开放权重LLM对于保护隐私的自动化具有吸引力，但在德语语境中往往难以保证编码准确性。本研究探讨了基于公共数据集的指令微调是否能提高开放权重LLM在德国肿瘤诊断文本上的编码准确性。评估使用来自当地肿瘤文档系统的已编码诊断作为测试数据。在系统的数据质量评估中，ICD-10编码性能的上限估计为精确推导60-79%，部分推导（仅三位字符代码）81-94%。作为训练数据，基于ICD-10-GM、ICD-O-3和OPS目录创建了超过50万个问答对。对来自Qwen、Llama和Mistral系列（7-70B参数）的八个开放权重模型进行了微调。ICD-10-GM的精确准确率从1.4-24%提升至41-58%，部分准确率从31-74%提升至73-83%。ICD-O-3形态学编码的准确率也有所提高，但起点和终点均明显较低，微调后精确准确率为22-40%，部分准确率为56-67%。所有模型的畸形代码输出均降至0%。肿瘤诊断识别率达到99%。准确率与模型大小呈正相关，但微调后小模型与大模型之间的差距缩小。Qwen3中的推理模式通常表现低于微调，且速度慢100倍以上。我们的研究结果凸显了利用公共目录构建指令数据集以改进LLM在医学文档任务中表现的潜力。完整的训练数据集和微调模型的最佳检查点可从https://huggingface.co/datasets/stefan-m-lenz/ICDOPS-QA-2024获取。
英文摘要：
Accurate coding of tumor diagnoses with ICD-10-GM and ICD-O-3 is essential for structured cancer documentation in Germany. Smaller open-weight LLMs are appealing for privacy-preserving automation but often struggle with coding accuracy in German-language contexts. This study investigates whether instruction-based fine-tuning on public datasets improves the coding accuracy of open-weight LLMs for German tumor diagnosis texts. The evaluation uses coded diagnoses from the local tumor documentation system as test data. In a systematic data quality assessment, the upper limit for ICD-10 coding performance was estimated at 60-79% for exact and 81-94% for partial (three-character codes only) derivation. As training data, over 500,000 question-answer pairs were created based on the ICD-10-GM, ICD-O-3, and OPS catalogues. Eight open-weight models from the Qwen, Llama, and Mistral families (7-70 B parameters) were fine-tuned. ICD-10-GM accuracy rose from 1.4-24% to 41-58%, and partial accuracy from 31-74% to 73-83%. The accuracy of ICD-O-3 topography coding also improved but started and remained considerably lower with an exact accuracy of 22-40% and a partial accuracy of 56-67% after fine-tuning. Malformed code outputs dropped to 0% for all models. Tumor-diagnosis recognition reached 99%. Accuracy correlated positively with model size, but gaps between small and large models narrowed after fine-tuning. The reasoning mode in Qwen3 generally yielded a lower performance than fine-tuning and was over 100 times slower. Our findings highlight the potential of leveraging public catalogues to build instruction datasets that improve LLMs in medical documentation tasks. The complete training dataset and the best-performing checkpoints of the fine-tuned models are available from https://huggingface.co/datasets/stefan-m-lenz/ICDOPS-QA-2024.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
