大型语言模型在出院后临床行动提取中的系统评估
Systematic Evaluation of Large Language Models for Post-Discharge Clinical Action Extraction
AI总结：
本文系统评估了零样本和少样本大型语言模型在安全关键的出院后临床行动提取中的表现，提出两阶段提取框架以提升临床任务的可操作性，并分析标注不一致性和模型推理与标注规范的对齐问题。
摘要：
本文评估了零样本和少样本大型语言模型（LLMs）在安全关键的出院后临床行动提取中的表现，特别关注转诊护理和出院患者安全。为管理临床文档的复杂性，我们引入了两阶段提取框架，通过分阶段提示策略将叙述性出院记录分解为细粒度、可操作的临床任务。我们的贡献包括对生成LLMs在临床行动提取中的系统评估，通用LLMs与任务特定监督BERT模型的详细比较，以及不同行动类别标注不一致性的分析。我们显示，当代LLMs在二元行动性检测上的性能可与或超过监督模型，而监督基线在细粒度多标签分类上仍保持明显优势，尽管没有任务特定的微调且在严格的数据隐私约束下。定性错误分析表明，许多失败源于模型推理与数据集标注规范的不一致，特别是在涉及隐含临床行动和刚性结构标注规则的情况下。这些结果表明，报告的性能反映了由于缺乏临床推理能力而产生的模型限制，这种限制未被普通标注所捕捉。没有理由的标签使区分临床推理失败与标注规范不匹配变得不可能。推进临床NLP需要记录为何特定跨度可操作的推理标注数据集，而非仅仅记录哪些跨度被标注，从而能够正确评估模型的临床理解能力。
英文摘要：
The work in this paper evaluates zero-shot and few-shot large language models (LLMs) for safety-critical clinical action extraction using the CLIP discharge-note dataset, with particular emphasis on transitions of care and post-discharge patient safety. To manage the complexity of clinical documentation, we introduce a two-stage extraction framework that decomposes discharge notes, that are written in narrative form, into fine-grained, explicitly actionable clinical tasks through a staged prompting strategy. Our contributions include a systematic assessment of generative LLMs for clinical action extraction, a detailed comparison between general-purpose LLMs and task-specific supervised BERT-based models, and an analysis of annotation inconsistencies across different action categories. We show that contemporary LLMs achieve performance comparable to or exceeding supervised models on binary actionability detection, while supervised baselines retain a meaningful advantage on fine-grained multi-label category classification, despite the absence of task-specific fine-tuning and under strict data-privacy constraints. Qualitative error analysis reveals that many failures stem from misalignment between model reasoning and dataset annotation conventions, particularly in cases involving implicit clinical actions and rigid structural labeling rules. These results indicate that reported performance reflects model limitations due to lack of clinical reasoning, that is not captured by plain annotations. Labels without rationales make it impossible to distinguish clinical reasoning failures from annotation convention mismatches. Advancing clinical NLP requires reasoning-annotated datasets that document why specific spans are actionable, not merely which spans were labeled, enabling proper evaluation of model clinical understanding.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
