SIFTING：一种基于LLM的新型框架，用于从临床自由文本报告中实现结构化且透明的信息提取，并应用于肺癌肿瘤分期
SIFTING: A Novel LLM-Based Framework for Structured and Transparent Information Extraction from Clinical Free-Text Reports, with Application to Tumor Staging in Lung Cancer
浏览论文内容
中文总结 AI 辅助
SIFTING框架结合LLM与分段处理和严格输出控制，实现临床文本的准确透明信息提取，在肺癌T分期任务中准确率达90%，媲美最先进模型和专家，且完全可追溯。
中文摘要 AI 辅助
背景：大型语言模型（LLM）在从临床自由文本文档中提取信息方面展现出潜力，但其输出通常是非结构化的且缺乏可追溯性，使得在临床工作流程中的验证和采用变得复杂。在本研究中，我们引入了SIFTING，一种基于LLM的框架，旨在解决这些不足。方法：SIFTING将LLM的语言理解能力与分段级处理以及具有严格输出控制的结构化提示相结合，将发现与源文本关联起来，从而实现准确且透明的信息提取。为了展示其能力，我们将该框架应用于从130份肺癌放射学报告中提取肿瘤T分期信息的任务（SIFTING-T-stage）。使用了开源LLM Llama-3.3-70B（35 GB）的紧凑型4位量化版本，在完全自托管的设置中运行，从而对数据和模型实现完全控制。性能根据由四位临床专家创建的参考标准进行评估，并与在传统单提示方法中使用的各种LLM进行比较，使用自助重采样来估计置信区间。结果：SIFTING-T-stage在参考标准上的准确率达到90%（95%置信区间：84-95）。我们发现其性能与甚至具有推理能力的最大的最先进LLM相当，并且与临床专家的结果可互换（p < 0.001），同时通过源文本引用提供了完全的追溯性。结论：SIFTING能够从临床自由文本文档中实现准确、结构化且可追溯的信息提取。它确保了数据控制、可重复性和可验证的输出，能够支持临床验证和工作流程集成。
英文摘要
Background: Large language models (LLMs) show promise for extracting information from clinical free-text documents, but their outputs are often unstructured and lack traceability, complicating validation and adoption in clinical workflows. In this work we introduce SIFTING, an LLM-based framework designed to address these shortcomings. Methods: SIFTING combines the language comprehension capabilities of LLMs with segment-level processing and structured prompts with strict output control, linking findings to the source text to enable both accurate and transparent information extraction. To demonstrate its capabilities, we applied the framework to the task of extracting tumor T-stage information from 130 lung cancer radiology reports (SIFTING-T-stage). A compact 4-bit quantized version of the open-source LLM Llama-3.3-70B (35 GB) was used in a fully self-hosted setup, providing full control over data and model. Performance was evaluated against a reference standard created by four clinical experts and compared with a range of LLMs as used in a conventional single-prompt approach, using bootstrap resampling to estimate confidence intervals. Results: SIFTING-T-stage achieved an accuracy of 90% (95% CI: 84-95) against the reference standard. We found its performance to be comparable to even the largest state-of-the-art LLMs with reasoning capabilities and to be interchangeable with clinical experts (p < 0.001), while at the same time offering full traceability through source text references. Conclusion: SIFTING enables accurate, structured, and traceable information extraction from clinical free-text documents. It ensures data control, reproducibility, and verifiable outputs that can support clinical validation and workflow integration.
发表机构
- Axana B.V.（阿克萨纳有限公司）
- University College London Hospitals NHS Foundation Trust（伦敦大学学院医院 NHS 基金会信托）
- University College London（伦敦大学学院）
- Mid & South Essex NHS Foundation Trust（中埃塞克斯及南埃塞克斯 NHS 基金会信托）
- NHS Greater Glasgow and Clyde（大格拉斯哥及克莱德 NHS）
机构由 AI 辅助整理，请以论文原文为准。
