3DReasonKnee：推进医学视觉语言模型中的接地推理
3DReasonKnee: Advancing Grounded Reasoning in Medical Vision Language Models
AI总结：
针对现有VLM无法在3D医学图像中接地推理的问题，构建首个3D接地推理数据集3DReasonKnee，含494k五元组，并建立ReasonKnee-Bench基准，推动医学AI向三维临床对齐决策发展。
摘要：
当前的视觉语言模型（VLM）难以在三维医学图像中定位解剖区域，并逐步对其进行推理，而这是现实世界诊断评估的关键要求。这一能力对于将模型输出与临床医生实际使用的诊断工作流程对齐至关重要，从而能够实现可信赖的临床医生与人工智能协作。现有的三维数据集提供了定位标签，但均不支持这种“接地推理”能力。为弥补这一空白，我们推出了3DReasonKnee，这是首个用于医学图像的三维接地推理数据集，提供了源自7,970个三维膝关节MRI体数据的494,000个高质量五元组。每个五元组包括：（1）三维MRI体数据，（2）针对特定解剖区域的诊断问题，（3）定位相关解剖结构的三维边界框，（4）临床医生生成的诊断推理步骤，明确详述三维推理过程，以及（5）相关解剖区域的结构化严重程度评估。3DReasonKnee的创建和验证涉及超过450小时的专家临床医生时间，用于手动分割MRI和生成推理链，确保了其卓越的质量和临床相关性。我们建立了ReasonKnee-Bench来评估定位和诊断准确性，从而深入了解VLM在解剖区域和诊断查询中执行接地和严重程度评估的能力。我们对五个最先进的VLM进行了基准测试，为ReasonKnee-Bench提供了基线性能。通过提供这一独特的专家标注三维推理路径资源，3DReasonKnee作为骨科医生诊断专业知识的存储库，并为推进多模态医学AI系统向三维、临床对齐、局部化决策能力发展提供了至关重要的测试平台。数据集可在以下网址找到：https://huggingface.co/datasets/rajpurkarlab/3DReasonKnee
英文摘要：
Current Vision-Language Models (VLMs) struggle to ground anatomical regions in 3D medical images and reason about them in a step-by-step manner, a key requirement of real-world diagnostic assessment. This ability is essential for aligning model outputs with the diagnostic workflows clinicians use in practice, enabling trustworthy clinician-AI collaboration. Existing 3D datasets provide localization labels, but none support this "grounded reasoning" ability. To address this gap, we introduce 3DReasonKnee, the first 3D grounded reasoning dataset for medical images, which provides 494k high-quality quintuples derived from 7,970 3D knee MRI volumes. Each quintuple includes: (1) the 3D MRI volume, (2) a diagnostic question targeting a specific anatomical region (3) a 3D bounding box localizing the relevant anatomical structures, (4) clinician-generated diagnostic reasoning steps that explicitly detail the 3D reasoning process, and (5) structured severity assessments for the relevant anatomical region. The creation and validation of 3DReasonKnee, involving over 450 hours of expert clinician time for manually segmenting MRIs and generating reasoning chains, ensures its superior quality and clinical relevance. We establish ReasonKnee-Bench to evaluate localization and diagnostic accuracy, providing insight into VLM ability to perform grounding and severity assessment across anatomical regions and diagnostic inquiries. We benchmark five state-of-the-art VLMs, providing baseline performance for ReasonKnee-Bench. By providing this unique resource of expert-annotated 3D reasoning pathways, 3DReasonKnee serves as a repository of orthopedic surgeons' diagnostic expertise and offers a vital testbed for advancing multimodal medical AI systems towards 3D, clinically aligned, localized decision-making capabilities. The dataset can be found in: https://huggingface.co/datasets/rajpurkarlab/3DReasonKnee
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
