面向3D计算机断层扫描的疾病中心视觉语言预训练与混合视觉编码
Disease-Centric Vision-Language Pretraining with Hybrid Visual Encoding for 3D Computed Tomography
AI总结：
提出一种结合CNN-ViT混合编码器、疾病级对比学习和诊断感知提示的视觉语言预训练框架，在CT-RATE和Rad-ChestCT上取得最优性能，并提升零样本诊断可靠性。
摘要：
视觉语言预训练（VLP）通过利用放射学报告作为丰富的文本监督，为通用医学人工智能带来了巨大希望，但现有方法因低效的视觉骨干和粗粒度的语义对齐而在3D CT成像中面临挑战。为解决这些问题，我们提出一个定制的VLP框架，包含三个关键组件：（1）CNN-ViT混合编码器，用3D CNN骨干替代ViT的补丁嵌入，以高效捕获局部解剖细节，同时保留全局注意力和与预训练跨模态先验的兼容性；（2）疾病级对比学习机制，使用可学习的查询令牌从完整报告中动态提取疾病特定语义，并与相应视觉特征对齐，从而在同一解剖区域内区分不同疾病；（3）诊断感知提示策略，采用真实临床短语和聚合的疾病原型来弥合预训练与推理之间的差距，并增强零样本诊断可靠性。我们的模型在CT-RATE（84.4% AUC，+5.1%）和Rad-ChestCT（75.4% AUC，+5.4%）上实现了最先进的性能，在具有挑战性的60疾病基准上获得了更大的提升（+9.8% AUC），并展示了向放射学报告生成的强迁移能力，突显了我们方法的通用性和临床实用性。
英文摘要：
Vision-language pre-training (VLP) holds great promise for general-purpose medical AI by leveraging radiology reports as rich textual supervision, yet existing methods struggle with 3D CT imaging due to inefficient visual backbones and coarse semantic alignment. To address these issues, we propose a tailored VLP framework featuring three key components: (1) a CNN-ViT hybrid encoder that replaces ViT's patch embedding with a 3D CNN backbone to efficiently capture local anatomical details while preserving global attention and compatibility with pre-trained cross-modal priors; (2) a disease-level contrastive learning mechanism using learnable query tokens to dynamically extract disease-specific semantics from full reports and align them with corresponding visual features, thereby disentangling distinct diseases within the same anatomical region; and (3) a diagnosis-aware prompt strategy that employs real clinical phrases and aggregated disease prototypes to bridge the pre-training-inference gap and enhance zero-shot diagnostic reliability. Our model achieves state-of-the-art performance on CT-RATE (84.4% AUC, +5.1%) and Rad-ChestCT (75.4% AUC, +5.4%), with even larger gains (+9.8% AUC) on a challenging 60-disease benchmark, and demonstrates strong transferability to radiology report generation, underscoring the generality and clinical utility of our approach.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
