面向胸部X光的基于锚定知识增强的医学视觉-语言预训练
Grounded Knowledge-Enhanced Medical Vision-Language Pre-training for Chest X-Ray
AI总结：
针对医学图文数据冗余导致对齐受损的问题，提出GK-MVLP框架，利用基于transformer的锚定知识增强模块实现医学知识与解剖区域的细粒度对齐，并在多类胸部X光下游任务上达到或超过最先进性能。
摘要：
医学基础模型有望通过提供稳健且通用的医学数据表示来革新医疗保健。医学视觉-语言预训练已成为一种很有前景的方法，用于学习医学图像和文本的领域通用表示。然而，当前利用医学图像与文本之间全局和局部对齐的算法可能会受到医学数据中冗余信息的损害。为解决这一问题，我们提出了一种用于胸部X光的基于锚定知识增强的医学视觉-语言预训练（GK-MVLP）框架。在该框架中，通过使用基于transformer的锚定知识增强模块，将医学知识锚定到适当的解剖区域，以实现医学知识的文本特征与相应解剖区域级视觉特征之间的细粒度对齐。GK-MVLP在下游图像理解任务（胸部X光疾病分类、疾病定位）、生成任务（报告生成）以及视觉-语言理解任务（医学视觉问答）上的性能与最先进方法相当或超过它们。我们的结果证明了引入锚定机制在消除偏差和改善胸部X光图像与放射学报告之间对齐方面的优势。
英文摘要：
Medical foundation models have the potential to revolutionize healthcare by providing robust and generalized representations of medical data. Medical vision-language pre-training has emerged as a promising approach for learning domain-general representations of medical image and text. Current algorithms that exploit global and local alignment between medical image and text could however be marred by redundant information in medical data. To address this issue, we propose a grounded knowledge-enhanced medical vision-language pre-training (GK-MVLP) framework for chest X-ray. In this framework, medical knowledge was grounded to the appropriate anatomical regions by using a transformer-based grounded knowledge-enhanced module for fine-grained alignment between textural features of medical knowledge and the corresponding anatomical region-level visual features. The performance of GK-MVLP was competitive with or exceeded the state of the art on downstream image understanding tasks (chest X-ray disease classification, disease localization), generative task (report generation), and vision-language understanding task (medical visual question-answering). Our results demonstrate the advantage of incorporating grounding mechanism to remove biases and improve the alignment between chest X-ray image and radiology report.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
