面向胃肠VQA中视觉 grounding 推理的多任务学习
Multi-Task Learning for Visually Grounded Reasoning in Gastrointestinal VQA
AI总结：
本文提出一个面向胃肠医学VQA的多任务框架，通过LoRA微调Florence-2并联合学习问答、解释生成与视觉grounding，在答案准确性和视觉定位上显著优于单任务基线。
摘要：
我们为 MediaEval Medico 2025 挑战赛提出了一个多任务框架，利用经过 LoRA 微调的 Florence-2 模型同时完成视觉问答（VQA）、解释生成和视觉 grounding。所提出的系统整合了三个精心构建的数据集：（1）用于问答学习的 Kvasir-VQA-x1；（2）一个经过合成扩充的解释数据集，提供结构化的医学推理；（3）将视觉特征与分割掩码关联起来的文本到区域配对。这种多任务设置使模型能够联合学习视觉 grounding、推理和解释，从而生成既准确又可解释的响应。大量评估表明，我们的方法在答案准确性和视觉定位方面均显著优于单任务基线，凸显了基于 grounding 的多任务学习在医学 VQA 应用中的有效性。
英文摘要：
We present a multi-task framework for the MediaEval Medico 2025 challenge, leveraging a LoRA-tuned Florence-2 model for simultaneous visual question answering (VQA), explanation generation, and visual grounding. The proposed system integrates three curated datasets: (1) Kvasir-VQA-x1 for question-answer learning, (2) a synthetically enriched explanation dataset offering structured medical reasoning, and (3) text-to-region pairs linking visual features with segmentation masks. This multi-task setup enables the model to jointly learn visual grounding, reasoning, and interpretation, producing responses that are both accurate and interpretable. Extensive evaluation demonstrates that our approach substantially improves over single-task baselines in both answer accuracy and visual localization, highlighting the effectiveness of grounded multi-task learning for medical VQA applications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
