GCS-M3VLT：用于视网膜图像描述的基于引导上下文自注意力的多模态医学视觉语言Transformer
GCS-M3VLT: Guided Context Self-Attention based Multi-modal Medical Vision Language Transformer for Retinal Image Captioning
AI总结：
针对有限标注数据下视网膜图像描述的挑战，提出基于引导上下文自注意力机制的多模态视觉语言模型GCS-M3VLT，有效融合视觉与文本特征，在DeepEyeNet数据集上实现BLEU@4提升0.023。
摘要：
视网膜图像分析对于眼部疾病的诊断和治疗至关重要，然而由于图像质量和病理的变异性，尤其是在标注数据有限的情况下，从图像生成准确的医学报告仍然具有挑战性。以往基于Transformer的模型在有限监督下难以整合视觉和文本信息。为此，我们提出了一种用于视网膜图像描述的新型视觉语言模型，通过引导上下文自注意力机制结合视觉和文本特征。即使在数据稀缺的场景下，该方法也能捕捉精细细节和全局临床上下文。在DeepEyeNet数据集上的大量实验表明，BLEU@4提升了0.023，并取得了显著的定性进步，突显了我们的模型在生成全面医学描述方面的有效性。
英文摘要：
Retinal image analysis is crucial for diagnosing and treating eye diseases, yet generating accurate medical reports from images remains challenging due to variability in image quality and pathology, especially with limited labeled data. Previous Transformer-based models struggled to integrate visual and textual information under limited supervision. In response, we propose a novel vision-language model for retinal image captioning that combines visual and textual features through a guided context self-attention mechanism. This approach captures both intricate details and the global clinical context, even in data-scarce scenarios. Extensive experiments on the DeepEyeNet dataset demonstrate a 0.023 BLEU@4 improvement, along with significant qualitative advancements, highlighting the effectiveness of our model in generating comprehensive medical captions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
