Tri-VQA: 面向多属性分析的三角推理医学视觉问答
Tri-VQA: Triangular Reasoning Medical Visual Question Answering for Multi-Attribute Analysis
AI总结：
针对现有医学视觉问答方法结果可信度不足的问题，提出基于因果效应的Tri-VQA三角推理框架，通过构建反向因果问题优化推理过程，在多中心内镜超声等医学VQA数据集上表现优于现有方法。
摘要：
医学视觉问答（Med-VQA）交叉领域是一项具有挑战性的研究课题，具备提升患者参与度、助力临床专家获取第二诊疗意见等优势。然而，现有基于联合嵌入的Med-VQA方法无法解释其给出的结果是基于正确推理还是巧合答案，这削弱了VQA答案的可信度。本文研究构建更具内聚性且更稳定的Med-VQA结构。受因果效应启发，我们提出了全新的三角推理VQA（Tri-VQA）框架，该框架从“为何得出该答案？”的视角构建反向因果问题，以阐明答案的来源并推动更合理的前向推理过程。我们在来自五个中心的内镜超声（EUS）多属性标注数据集上评估所提方法，并在多个医学VQA数据集上进行测试。实验结果表明，我们的方法优于现有方法。我们的代码和预训练模型可在https://anonymous.4open.science/r/Tri_VQA获取。
英文摘要：
The intersection of medical Visual Question Answering (Med-VQA) is a challenging research topic with advantages including patient engagement and clinical expert involvement for second opinions. However, existing Med-VQA methods based on joint embedding fail to explain whether their provided results are based on correct reasoning or coincidental answers, which undermines the credibility of VQA answers. In this paper, we investigate the construction of a more cohesive and stable Med-VQA structure. Motivated by causal effect, we propose a novel Triangular Reasoning VQA (Tri-VQA) framework, which constructs reverse causal questions from the perspective of "Why this answer?" to elucidate the source of the answer and stimulate more reasonable forward reasoning processes. We evaluate our method on the Endoscopic Ultrasound (EUS) multi-attribute annotated dataset from five centers, and test it on medical VQA datasets. Experimental results demonstrate the superiority of our approach over existing methods. Our codes and pre-trained models are available at https://anonymous.4open.science/r/Tri_VQA.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
