聚焦图像：用于胸部X光诊断与报告生成的注视监督多模态学习
Eyes on the Image: Gaze Supervised Multimodal Learning for Chest X-ray Diagnosis and Report Generation
AI总结：
本研究基于MIMIC-Eye数据集构建两阶段多模态框架，引入注视监督提升胸部X光诊断准确率与报告空间可解释性，为该领域提供了可复现的新基准。
摘要：
医学视觉-语言模型仍难以匹配放射科医生的注意力，也难以用明确的空间定位表述检查结果。我们针对这一缺口，构建了基于MIMIC-Eye数据集的胸部X光解读两阶段多模态框架。第一阶段引入注视标记分类器，融合图像块、边界框掩码、转录嵌入及放射科医生的注视点；采用课程调度、信任校准的复合损失监督注视标记，提升了准确率与空间对齐度。添加注视监督后，AUC提升4.4%，F1提升13.3%，皮尔逊相关系数升至0.306，证实了与临床相关的聚焦效果。第二阶段将分类器预测转化为特定区域的诊断语句：提取置信度加权关键词，通过专家词典映射至17个胸部区域，再经提示式大语言模型扩展，较关键词基线提升了临床术语BERTScore与ROUGE分数。所有组件均可切换以进行消融实验，完整流程可复现，为可解释、感知注视的胸部X光分析提供了新基准；整合眼动追踪信号可显著提升诊断准确率及生成报告的透明度。
英文摘要：
Medical vision-language models still struggle to match radiologists' attention and to verbalize findings with explicit spatial grounding. We address this gap with a two-stage multimodal framework for chest X-ray interpretation built on the MIMIC-Eye dataset. In the first stage introduces a gaze-token classifier that fuses image patches, bounding-box masks, transcription embeddings, and radiologist fixations. A curriculum-scheduled, trust-calibrated composite loss supervises the gaze token, boosting both accuracy and spatial alignment. Adding fixation supervision raises AUC 4.4% and F1 13.3%, and Pearson correlation rises to 0.306, confirming clinically relevant focus. In stage 2, classifier predictions are translated into region-specific diagnostic sentences. Confidence-weighted keywords are extracted, mapped to 17 thoracic regions through an expert dictionary, and expanded with a prompted large language model, boosting clinical-term BERTScore and ROUGE scores over keyword baselines. All components are toggle-able for ablation, and the full pipeline is reproducible, offering a new benchmark for interpretable, gaze-aware chest-X-ray analysis. Integrating eye-tracking signals demonstrably enhances both diagnostic accuracy and the transparency of generated reports.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
