为什么链式思维在临床文本理解中失效
Why Chain of Thought Fails in Clinical Text Understanding
AI总结：
本研究发现链式思维在临床文本理解中导致性能下降，揭示了其在临床任务中可靠性与可解释性的矛盾。
摘要：
大型语言模型（LLMs）正越来越多地应用于临床护理领域，这是一个需要准确性和透明推理以确保安全可靠部署的关键领域。链式思维（CoT）提示法，通过引导逐步推理，已在广泛的任务中展示了性能和可解释性的提升。然而，在临床情境中的有效性仍鲜有探索，特别是在电子健康记录（EHRs）中，这是临床文档的主要来源，通常冗长、碎片化且嘈杂。在本工作中，我们首次进行了大规模的系统研究，评估了95种先进的LLMs在87个真实世界临床文本任务上的表现，涵盖9种语言和8种任务类型。与之前其他领域的研究发现相反，我们发现86.3%的模型在CoT设置下表现出一致的性能下降。更强大的模型相对稳健，而较弱的模型则有显著下降。为了更好地表征这些影响，我们对推理长度、医学概念对齐和错误特征进行了细致分析，利用LLM作为评判者和临床专家评估。我们的结果揭示了CoT在临床情境中何时以及为何失效的系统性模式，突显了一个关键矛盾：CoT增强了可解释性，但在临床文本任务中可能损害可靠性。本工作为LLM的临床推理策略提供了实证基础，突显了透明和可信赖方法的必要性。
英文摘要：
Large language models (LLMs) are increasingly being applied to clinical care, a domain where both accuracy and transparent reasoning are critical for safe and trustworthy deployment. Chain-of-thought (CoT) prompting, which elicits step-by-step reasoning, has demonstrated improvements in performance and interpretability across a wide range of tasks. However, its effectiveness in clinical contexts remains largely unexplored, particularly in the context of electronic health records (EHRs), the primary source of clinical documentation, which are often lengthy, fragmented, and noisy. In this work, we present the first large-scale systematic study of CoT for clinical text understanding. We assess 95 advanced LLMs on 87 real-world clinical text tasks, covering 9 languages and 8 task types. Contrary to prior findings in other domains, we observe that 86.3\% of models suffer consistent performance degradation in the CoT setting. More capable models remain relatively robust, while weaker ones suffer substantial declines. To better characterize these effects, we perform fine-grained analyses of reasoning length, medical concept alignment, and error profiles, leveraging both LLM-as-a-judge evaluation and clinical expert evaluation. Our results uncover systematic patterns in when and why CoT fails in clinical contexts, which highlight a critical paradox: CoT enhances interpretability but may undermine reliability in clinical text tasks. This work provides an empirical basis for clinical reasoning strategies of LLMs, highlighting the need for transparent and trustworthy approaches.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
