CheXthought:一个包含临床推理链和视觉注意力的全球多模态数据集,用于胸部X光解读
CheXthought: A global multimodal dataset of clinical chain-of-thought reasoning and visual attention for chest X-ray interpretation
AI总结:
CheXthought数据集通过501名放射科医生的50312例多读X光片,提供了103592条推理链和6609082条同步视觉注意力标注,展示了临床推理模式,并在事实准确性、空间定位、病理分类、视觉忠实度、时间推理和不确定性沟通等方面提升了视觉语言模型性能。
摘要:
胸部X光解读是医学中最常见的诊断任务之一,也是AI发展的主要目标,但当前的视觉语言模型主要是在配对图像和报告的数据集上训练,而非临床推理背后的认知过程和视觉注意力。本文介绍了CheXthought,一个包含103592条推理链和6609082条同步视觉注意力标注的全球多模态资源,涵盖50312例多读X光片,来自71个国家的501名放射科医生。我们的分析揭示了专家如何部署不同的视觉搜索策略、整合临床背景和沟通不确定性。我们展示了CheXthought在四个维度上的临床价值:首先,CheXthought的推理显著优于最先进的视觉语言模型推理链在事实准确性和空间定位方面;其次,作为推理时间提示的视觉注意力数据恢复了遗漏的发现并显著减少了幻觉;第三,使用CheXthought数据训练的视觉语言模型在病理分类、视觉忠实度、时间推理和不确定性沟通方面表现更强;第四,利用CheXthought的多读者标注,我们能够从图像直接预测人与人之间的以及人与AI之间的分歧,从而实现透明的沟通病例难度、不确定性和模型可靠性。这些发现确立了CheXthought作为推动多模态临床推理和更透明、可解释的视觉语言模型发展的资源。
英文摘要:
Chest X-ray interpretation is one of the most frequently performed diagnostic tasks in medicine and a primary target for AI development, yet current vision-language models are primarily trained on datasets of paired images and reports, not the cognitive processes and visual attention that underlie clinical reasoning. Here, we present CheXthought, a global, multimodal resource containing 103,592 chain-of-thought reasoning traces and 6,609,082 synchronized visual attention annotations across 50,312 multi-read chest X-rays from 501 radiologists in 71 countries. Our analysis reveals clinical reasoning patterns in how experts deploy distinct visual search strategies, integrate clinical context, and communicate uncertainty. We demonstrate the clinical utility of CheXthought across four dimensions. First, CheXthought reasoning significantly outperforms state-of-the-art vision-language model chain-of-thought in factual accuracy and spatial grounding. Second, visual attention data used as an inference-time hint recovers missed findings and significantly reduces hallucinations. Third, vision-language models trained on CheXthought data achieve significantly stronger pathology classification, visual faithfulness, temporal reasoning and uncertainty communication. Fourth, leveraging CheXthought's multi-reader annotations, we predict both human-human and human-AI disagreement directly from an image, enabling transparent communication of case difficulty, uncertainty and model reliability. These findings establish CheXthought as a resource for advancing multimodal clinical reasoning and the development of more transparent, interpretable vision-language models.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。