UnICLAM：面向统一且可解释医学视觉问答的、采用对抗掩码的对比表征学习
UnICLAM:Contrastive Representation Learning with Adversarial Masking for Unified and Interpretable Medical Vision Question Answering
AI总结：
本文提出 UnICLAM，通过渐进式软参数共享与统一对抗掩码对比学习解决医学 VQA 的跨模态对齐问题，并提升可解释性与少样本诊断性能。
摘要：
医学视觉问答（Medical-VQA）旨在回答有关放射学图像的临床问题，从而为医生提供决策选择方面的协助。然而，当前的 Medical-VQA 模型通过将视觉编码器与文本编码器置于两个彼此分离的空间中来学习跨模态表征，这会导致间接的语义对齐。本文提出 UnICLAM，这是一种通过带对抗掩码的对比表征学习实现的统一且可解释的 Medical-VQA 模型。具体而言，为学习对齐的图像-文本表征，我们首先采用渐进式软参数共享策略建立统一的双流预训练结构。在技术上，所提出的策略学习一种约束，使视觉编码器和文本编码器在同一空间中彼此接近，并随着层数增加而逐渐放宽该约束。此外，为掌握统一的语义表征，我们将对抗掩码数据增强以统一方式扩展到视觉和文本的对比表征学习中。具体而言，编码器训练会最小化原始样本与掩码样本之间的距离，而对抗掩码模块则持续进行对抗学习，以反向方式最大化该距离。此外，我们还直观地对统一对抗掩码增强模型作进一步探索，该模型以出色的性能和效率提升了潜在的事前可解释性。在 VQA-RAD 和 SLAKE 公开基准上的实验结果表明，UnICLAM 优于现有 11 个最先进的 Medical-VQA 模型。更重要的是，我们额外讨论了 UnICLAM 在诊断心力衰竭方面的性能，验证了 UnICLAM 在实际疾病诊断中展现出优越的少样本适应性能。
英文摘要：
Medical Visual Question Answering (Medical-VQA) aims to to answer clinical questions regarding radiology images, assisting doctors with decision-making options. Nevertheless, current Medical-VQA models learn cross-modal representations through residing vision and texture encoders in dual separate spaces, which lead to indirect semantic alignment. In this paper, we propose UnICLAM, a Unified and Interpretable Medical-VQA model through Contrastive Representation Learning with Adversarial Masking. Specifically, to learn an aligned image-text representation, we first establish a unified dual-stream pre-training structure with the gradually soft-parameter sharing strategy. Technically, the proposed strategy learns a constraint for the vision and texture encoders to be close in a same space, which is gradually loosened as the higher number of layers. Moreover, for grasping the unified semantic representation, we extend the adversarial masking data augmentation to the contrastive representation learning of vision and text in a unified manner. Concretely, while the encoder training minimizes the distance between original and masking samples, the adversarial masking module keeps adversarial learning to conversely maximize the distance. Furthermore, we also intuitively take a further exploration to the unified adversarial masking augmentation model, which improves the potential ante-hoc interpretability with remarkable performance and efficiency. Experimental results on VQA-RAD and SLAKE public benchmarks demonstrate that UnICLAM outperforms existing 11 state-of-the-art Medical-VQA models. More importantly, we make an additional discussion about the performance of UnICLAM in diagnosing heart failure, verifying that UnICLAM exhibits superior few-shot adaption performance in practical disease diagnosis.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
