CORTEX:用于可信3D胸部CT多模态大语言模型的结构化推理基准
CORTEX: A Structured Reasoning Benchmark for Trustworthy 3D Chest CT MLLMs
AI总结:
提出CORTEX基准,通过四阶段诊断推理轨迹和阶段级评估协议,为3D胸部CT多模态大语言模型提供结构化监督与验证,包含76,177个验证推理轨迹。
摘要:
多模态大语言模型(MLLMs)在医学影像中的推理展现出强大潜力。然而,这种推理通常以自由文本形式呈现,仅通过最终答案评判,导致难以解释和验证,尤其是在3D放射学中,诊断应可追溯到扫描中的证据。现有的胸部CT问答数据集通过将专家放射学报告简化为仅答案对,进一步加剧了这一问题,丢弃了将发现与结论联系起来的推理,并省略了临床医生依赖的患者病史。因此,具备推理能力的3D胸部CT MLLMs仍然遥不可及,因为既缺乏训练所需的结构化监督,也缺乏验证其推理所需的协议。我们提出CORTEX(临床组织推理与结构化解释),一个用于3D胸部CT的结构化推理基准。对于每个问题,CORTEX将缺失的推理恢复为四阶段诊断轨迹,镜像放射科医生的工作流程:任务理解、视觉观察、诊断推理和答案合成。我们利用具有广泛医学和通用领域知识的前沿大语言模型生成这些轨迹,然后通过结合自动评分标准与专家放射科医生审查的阶段级评估协议进行筛选和验证。关键的是,推理结构和评估标准均与临床医生紧密合作设计。CORTEX基于CT-RATE(一个大型公开可用但无推理标注的胸部CT数据集),包含76,177个经过验证的推理轨迹,涵盖开放式VQA、封闭式VQA和报告生成,为构建和评估3D胸部CT可信推理模型提供了所需的结构化监督和阶段级评估协议。我们的数据集和评估代码将在接收后公开。
英文摘要:
Reasoning in multimodal large language models (MLLMs) has shown strong promise in medical imaging. However, this reasoning is usually free-form text judged only by its final answer, making it hard to interpret and verify, especially in 3D radiology, where a diagnosis should be traceable to evidence in the scan. Existing chest CT question-answering datasets compound this by reducing expert radiology reports to answer-only pairs, dropping the reasoning that links findings to conclusions and omitting the patient history clinicians rely on. As a result, reasoning-capable 3D chest CT MLLMs remain out of reach, as neither the structured supervision needed to train them nor the protocol needed to verify their reasoning yet exists. We introduce CORTEX (Clinically Organized Reasoning and sTructured EXplanation), a structured reasoning benchmark for 3D chest CT. For each question, CORTEX restores the missing reasoning as a four-stage diagnostic trace mirroring a radiologist's workflow: task understanding, visual observation, diagnostic reasoning, and answer synthesis. We generate these traces using frontier large language models with broad medical and general-domain knowledge, then filter and verify them with a stage-level evaluation protocol combining automated rubric scoring with expert radiologist review. Crucially, both the reasoning structure and evaluation rubrics are designed in close collaboration with clinicians. Built on CT-RATE, a large, publicly available chest CT dataset without reasoning annotations, CORTEX comprises 76,177 validated reasoning traces across open-ended VQA, closed-ended VQA, and report generation, providing both the structured supervision and the stage-level evaluation protocol needed to build and evaluate trustworthy reasoning models for 3D chest CT. Our dataset and evaluation code is available at https://huggingface.co/datasets/aneesurhashmi/cortex
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。