面向位置的预训练方法用于医学差异视觉问答
Location-Aware Pretraining for Medical Difference Visual Question Answering
AI总结：
本文提出一种面向位置的预训练框架，结合AREF、GCAP和CAREF任务，提升医学差异VQA中细粒度空间视觉表征能力，实现胸部X光图像中临床相关变化的准确识别与推理。
摘要：
差分医学VQA模型通过比较多张图像来识别具有临床意义的变化，并依赖视觉编码器捕捉细粒度的视觉差异，以反映放射科医生的比较诊断流程。然而，使用标准对比或分类目标训练的视觉编码器往往无法捕捉到区分真实疾病进展与获取相关变异所需的细微变化。为了解决这一限制，我们引入了一种面向位置的预训练框架，结合自动指代表达（AREF）、基于语境的描述（GCAP）和条件自动指代表达（CAREF）。这些任务促进了细粒度、空间接地的视觉表征学习。当与语言模型结合时，我们的方法在医学差异VQA中实现了最先进的性能，能够准确识别和推理胸部X光图像中的临床相关变化。
英文摘要：
Differential medical VQA models compare multiple images to identify clinically meaningful changes and rely on vision encoders to capture fine-grained visual differences that reflect radiologists' comparative diagnostic workflows. However, vision encoders trained using standard contrastive or classification objectives often fail to capture the subtle variations needed to distinguish true disease progression from acquisition-related variability. To address this limitation, we introduce a location-aware pretraining framework that incorporates automatic referring expressions (AREF), grounded captioning (GCAP), and conditional automatic referring expressions (CAREF). These tasks promote the learning of fine-grained, spatially grounded visual representations. When integrated with a language model, our approach achieves state-of-the-art performance on medical difference VQA by accurately identifying and reasoning about clinically relevant changes in chest X-ray images.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
