像放射科医生一样观察：面向胸部X光片的上下文和目光引导的视觉-语言预训练
Seeing Like Radiologists: Context- and Gaze-Guided Vision-Language Pretraining for Chest X-rays
AI总结：
本文提出CoGaze框架，通过整合临床上下文和放射科医生目光引导，提升胸部X光片的视觉-语言预训练效果，实验显示在多个任务上均优于现有方法。
摘要：
尽管医学视觉-语言预训练取得了进展，现有模型仍难以捕捉诊断流程：放射影像通常被视为无上下文的图像，而放射科医生的目光——视觉推理的关键线索——仍被现有方法忽视。这些限制阻碍了疾病特定模式的建模和跨模态对齐。为此，我们引入CoGaze，一种面向胸部X光片的上下文和目光引导的视觉-语言预训练框架。我们首先提出一种上下文融入的视觉编码器，建模放射科医生如何整合临床上下文——包括患者病史、症状和诊断意图——以引导诊断推理。然后提出多级监督范式，通过混合正对比学习强制内在和跨模态语义对齐，通过疾病意识的跨模态表示学习注入诊断先验，并通过放射科医生的目光作为概率先验引导注意力集中在诊断显著区域。大量实验表明，CoGaze在多个任务上均优于现有方法，实现了在自由文本和结构化报告生成任务中分别达到+2.0% CheXbertF1和+1.2% BLEU2，零样本分类中达到+23.2% AUROC，图像-文本检索中达到+12.2% Precision@1。代码可在https://github.com/mk-runner/CoGaze获取。
英文摘要：
Despite recent advances in medical vision-language pretraining, existing models still struggle to capture the diagnostic workflow: radiographs are typically treated as context-agnostic images, while radiologists' gaze -- a crucial cue for visual reasoning -- remains largely underexplored by existing methods. These limitations hinder the modeling of disease-specific patterns and weaken cross-modal alignment. To bridge this gap, we introduce CoGaze, a Context- and Gaze-guided vision-language pretraining framework for chest X-rays. We first propose a context-infused vision encoder that models how radiologists integrate clinical context -- including patient history, symptoms, and diagnostic intent -- to guide diagnostic reasoning. We then present a multi-level supervision paradigm that (1) enforces intra- and inter-modal semantic alignment through hybrid-positive contrastive learning, (2) injects diagnostic priors via disease-aware cross-modal representation learning, and (3) leverages radiologists' gaze as probabilistic priors to guide attention toward diagnostically salient regions. Extensive experiments demonstrate that CoGaze consistently outperforms state-of-the-art methods across diverse tasks, achieving up to +2.0% CheXbertF1 and +1.2% BLEU2 for free-text and structured report generation, +23.2% AUROC for zero-shot classification, and +12.2% Precision@1 for image-text retrieval. Code is available at https://github.com/mk-runner/CoGaze.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
