See-in-Pairs: 用于医学诊断的参考图像引导的比较视觉-语言模型
See-in-Pairs: Reference Image-Guided Comparative Vision-Language Models for Medical Diagnosis
AI总结：
See-in-Pairs通过引入参考图像引导的比较机制，提升医学视觉-语言模型的诊断性能，增强样本效率和视觉-文本对齐。
摘要：
医学图像诊断具有挑战性，因为许多疾病与正常解剖结构相似，并且在患者之间表现出显著的变异性。临床医生通常依赖比较诊断，例如通过参考跨患者健康对照图像来识别微妙但具有临床意义的异常。尽管健康参考图像在实践中很丰富，但现有的医学视觉-语言模型（VLMs）主要在单张图像或单系列设置下运行，并且缺乏明确的比较诊断机制。本文探讨了是否可以将临床动机的比较纳入VLM中以提高其性能。我们证明，为VLM提供查询图像和匹配的健康参考图像，并辅以跨患者比较提示，可以显著提高诊断性能。这种性能可以通过在少量数据上进行轻量级监督微调（SFT）进一步增强。同时，我们评估了多种选择参考图像的策略，包括随机采样、人口统计学属性匹配、基于嵌入的检索和跨中心选择，并发现所有设置下都表现出一致的强性能。最后，我们调查了比较诊断为何有效的理论原因，并观察到样本效率的提高以及视觉和文本表示之间的对齐更加紧密。我们的发现突显了基于比较的诊断的临床相关性，提供了将参考图像纳入VLM的实用策略，并展示了在多样化的医学成像任务中性能的提升。
英文摘要：
Medical image diagnosis is challenging because many diseases resemble normal anatomy and exhibit substantial interpatient variability. Clinicians routinely rely on comparative diagnosis, such as referencing cross-patient healthy control images to identify subtle but clinically meaningful abnormalities. Although healthy reference images are abundant in practice, existing medical vision-language models (VLMs) primarily operate in a single-image or single-series setting and lack explicit mechanisms for comparative diagnosis. This work investigates whether incorporating clinically motivated comparison can enhance VLM performance. We show that providing VLMs with both a query image and a matched healthy reference image, accompanied by cross-patient comparative prompts, significantly improves diagnostic performance. This performance can be further augmented by lightweight supervised fine-tuning (SFT) on a small amount of data. At the same time, we evaluate multiple strategies for selecting reference images, including random sampling, demographic attribute matching, embedding-based retrieval, and cross-center selection, and find consistently strong performance across all settings. Finally, we investigate why comparative diagnosis is effective theoretically, and observe improved sample efficiency and tighter alignment between visual and textual representations. Our findings highlight the clinical relevance of comparison-based diagnosis, provide practical strategies for incorporating reference images into VLMs, and demonstrate improved performance across diverse medical imaging tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
